uppy/packages/@uppy/utils/src/generateFileID.js
Renée Kooi 1463ee79ce
Improve performance of adding and removing files (#1949)
* core: add an addFiles() method that only updates state once

Previously, adding 1300-ish files took around 260ms, and looked like
this in Firefox's performance tab:

![Flamegraph](https://i.imgur.com/08veuoU.png)

All of the downward peaks are `setState()` calls and GC.

Now it takes around 60ms, and looks like this:

![Flamegraph](https://i.imgur.com/xFdwVBV.png)

Here, most of the time is spent generating file IDs and guessing file
types. Those would be areas to look at next.

* dashboard: prevent frequent state update if nothing changed

After the last commit, `addFiles()` still spends a lot of time in
`emit()` and `hideAllPanels()`. The reason is that `addFiles()` still
emits all the hundreds of file-added events, and the Dashboard responds
to each with `hideAllPanels()`, which does a state update. But this all
happens synchronously, and the state almost certainly did not change
since the last `file-added` event that fired a millisecond ago.

This adds a check to avoid the state update if it is not necessary.

![Flamegraph](https://i.imgur.com/KhuD035.png)

Adding 1300 files takes about 40ms now.

With this change, the `addFiles()` call is no longer the slowest
part—now preact rendering all the items is!

* utils: optimize generateFileID and getFileNameAndExtension

Replaces some clever things with more mundane and faster things!

Now, generateFileID is a bunch of string concatenations.
Now, getFileNameAndExtension uses `lastIndexOf()` instead of a regex.

![Flamegraph](https://i.imgur.com/ZQ1IhxI.png)

Adding 1300 files takes about 25ms.

* dashboard: use preact-virtual-list

* thumbnail-generator: add `lazy` option

* dashboard: request thumbnails once file item renders the first time

* dashboard: fork preact-virtual-list

* core: add removeFiles() to remove files in bulk

* Implement removeFile() in terms of removeFiles()

* thumbnail-generator: only queue files that can be previewed

* rename size constants to accommodate WIDTH/HEIGHT

* Use new uppy.addFiles in DragDrop and FileInput

* utils: fix getFileNameAndExtension() type

* Rip out the lazy thumbnail generation

* Rip out virtualization.

* Remove virtualization leftovers

* tell future people that this is intentionally verbose

* Update package-lock.json

* henlo i am spell

* Make `addFiles()` respect maxNumberOfFiles

* core: show an informer error if some files fail in bulk add

* locales: fix quotes to make build:locale-pack happy

Co-authored-by: Artur Paikin <artur@arturpaikin.com>
2020-01-13 12:29:08 +01:00

45 lines
1.1 KiB
JavaScript

/**
* Takes a file object and turns it into fileID, by converting file.name to lowercase,
* removing extra characters and adding type, size and lastModified
*
* @param {object} file
* @returns {string} the fileID
*/
module.exports = function generateFileID (file) {
// It's tempting to do `[items].filter(Boolean).join('-')` here, but that
// is slower! simple string concatenation is fast
let id = 'uppy'
if (typeof file.name === 'string') {
id += '-' + encodeFilename(file.name.toLowerCase())
}
if (file.type !== undefined) {
id += '-' + file.type
}
if (file.meta && typeof file.meta.relativePath === 'string') {
id += '-' + encodeFilename(file.meta.relativePath.toLowerCase())
}
if (file.data.size !== undefined) {
id += '-' + file.data.size
}
if (file.data.lastModified !== undefined) {
id += '-' + file.data.lastModified
}
return id
}
function encodeFilename (name) {
let suffix = ''
return name.replace(/[^A-Z0-9]/ig, (character) => {
suffix += '-' + encodeCharacter(character)
return '/'
}) + suffix
}
function encodeCharacter (character) {
return character.charCodeAt(0).toString(32)
}