uppy/packages/@uppy/utils/src
Renée Kooi 1463ee79ce
Improve performance of adding and removing files (#1949)
* core: add an addFiles() method that only updates state once

Previously, adding 1300-ish files took around 260ms, and looked like
this in Firefox's performance tab:

![Flamegraph](https://i.imgur.com/08veuoU.png)

All of the downward peaks are `setState()` calls and GC.

Now it takes around 60ms, and looks like this:

![Flamegraph](https://i.imgur.com/xFdwVBV.png)

Here, most of the time is spent generating file IDs and guessing file
types. Those would be areas to look at next.

* dashboard: prevent frequent state update if nothing changed

After the last commit, `addFiles()` still spends a lot of time in
`emit()` and `hideAllPanels()`. The reason is that `addFiles()` still
emits all the hundreds of file-added events, and the Dashboard responds
to each with `hideAllPanels()`, which does a state update. But this all
happens synchronously, and the state almost certainly did not change
since the last `file-added` event that fired a millisecond ago.

This adds a check to avoid the state update if it is not necessary.

![Flamegraph](https://i.imgur.com/KhuD035.png)

Adding 1300 files takes about 40ms now.

With this change, the `addFiles()` call is no longer the slowest
part—now preact rendering all the items is!

* utils: optimize generateFileID and getFileNameAndExtension

Replaces some clever things with more mundane and faster things!

Now, generateFileID is a bunch of string concatenations.
Now, getFileNameAndExtension uses `lastIndexOf()` instead of a regex.

![Flamegraph](https://i.imgur.com/ZQ1IhxI.png)

Adding 1300 files takes about 25ms.

* dashboard: use preact-virtual-list

* thumbnail-generator: add `lazy` option

* dashboard: request thumbnails once file item renders the first time

* dashboard: fork preact-virtual-list

* core: add removeFiles() to remove files in bulk

* Implement removeFile() in terms of removeFiles()

* thumbnail-generator: only queue files that can be previewed

* rename size constants to accommodate WIDTH/HEIGHT

* Use new uppy.addFiles in DragDrop and FileInput

* utils: fix getFileNameAndExtension() type

* Rip out the lazy thumbnail generation

* Rip out virtualization.

* Remove virtualization leftovers

* tell future people that this is intentionally verbose

* Update package-lock.json

* henlo i am spell

* Make `addFiles()` respect maxNumberOfFiles

* core: show an informer error if some files fail in bulk add

* locales: fix quotes to make build:locale-pack happy

Co-authored-by: Artur Paikin <artur@arturpaikin.com>
2020-01-13 12:29:08 +01:00
..
getDroppedFiles getDroppedFiles.js: handle exceptions better (#1797) 2019-08-30 14:45:21 +03:00
canvasToBlob.js eslint jsdoc fixes 2019-06-25 18:27:48 +03:00
dataURItoBlob.js Update eslint-plugin-compat and eslint-plugin-jest 2019-05-20 11:29:10 +02:00
dataURItoBlob.test.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
dataURItoFile.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
dataURItoFile.test.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
emitSocketProgress.js Upgrade eslint. 2019-05-20 11:28:44 +02:00
EventTracker.js Use more cancellation-friendly strategy for limit: N uploads (#1736) 2019-09-25 15:21:17 +02:00
findAllDOMElements.js eslint jsdoc fixes 2019-06-25 18:27:48 +03:00
findDOMElement.js eslint jsdoc fixes 2019-06-25 18:27:48 +03:00
FOCUSABLE_ELEMENTS.js Feature/accessibility (#1507) 2019-06-11 19:10:32 +03:00
generateFileID.js Improve performance of adding and removing files (#1949) 2020-01-13 12:29:08 +01:00
generateFileID.test.js core: Don’t add/overwrite existing files, allow duplicates from different folders (#1767) 2019-09-19 16:43:51 +03:00
getBytesRemaining.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
getBytesRemaining.test.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
getETA.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
getETA.test.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
getFileNameAndExtension.js Improve performance of adding and removing files (#1949) 2020-01-13 12:29:08 +01:00
getFileNameAndExtension.test.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
getFileType.js core: Added heic file type, refactor getFileType (#1734) 2019-07-17 19:54:06 +02:00
getFileType.test.js Fixed getFileType() when passed a file with an upper case extension 2018-11-23 15:13:08 +01:00
getFileTypeExtension.js Support mime types with additional data, fixes #1074 2018-10-08 15:26:32 +02:00
getFileTypeExtension.test.js Support mime types with additional data, fixes #1074 2018-10-08 15:26:32 +02:00
getSocketHost.js companion-client: remove the use of window.location 2019-04-01 14:44:03 +01:00
getSocketHost.test.js Upgrade eslint. 2019-05-20 11:28:44 +02:00
getSpeed.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
getSpeed.test.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
getTimeStamp.js Update eslint to v6 (#1777) 2019-08-13 21:45:30 +03:00
hasProperty.js Update eslint to v6 (#1777) 2019-08-13 21:45:30 +03:00
index.js Add @uppy/utils skeleton. 2018-06-14 16:31:19 +02:00
isDOMElement.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
isDragDropSupported.js eslint jsdoc fixes 2019-06-25 18:27:48 +03:00
isObjectURL.js eslint jsdoc fixes 2019-06-25 18:27:48 +03:00
isObjectURL.test.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
isPreviewSupported.js thumbnail-generator: Add webp to the list of supported types (#1961) 2019-12-04 11:07:47 +01:00
isPreviewSupported.test.js thumbnail-generator: Add webp to the list of supported types (#1961) 2019-12-04 11:07:47 +01:00
isTouchDevice.js Update eslint-plugin-compat and eslint-plugin-jest 2019-05-20 11:29:10 +02:00
isTouchDevice.test.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
microtip.scss Move stylesheets into packages. 2018-06-18 12:35:47 +02:00
mimeTypes.js Update eslint to v6 (#1777) 2019-08-13 21:45:30 +03:00
prettyBytes.js utils: prettyBytes 1000 --> 1024 (#1732) 2019-07-16 15:21:47 +02:00
prettyBytes.test.js utils: prettyBytes 1000 --> 1024 (#1732) 2019-07-16 15:21:47 +02:00
prettyETA.js Commit improved code formatting suggestion 2019-04-28 13:16:01 +01:00
prettyETA.test.js Status Bar: Hide seconds if more than 1 hour 2019-04-27 21:56:42 +01:00
ProgressTimeout.js Use more cancellation-friendly strategy for limit: N uploads (#1736) 2019-09-25 15:21:17 +02:00
RateLimitedQueue.js utils: accept sync functions in wrapPromiseFunction() (#1910) 2019-10-25 11:07:06 +02:00
RateLimitedQueue.test.js utils: accept sync functions in wrapPromiseFunction() (#1910) 2019-10-25 11:07:06 +02:00
sampleImageDataURI.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
secondsToTime.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
secondsToTime.test.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
settle.js Move utils files to @uppy/utils package. 2018-06-14 16:31:19 +02:00
settle.test.js Update eslint to v6 (#1777) 2019-08-13 21:45:30 +03:00
toArray.js Update eslint to v6 (#1777) 2019-08-13 21:45:30 +03:00
toArray.test.js Update eslint to v6 (#1777) 2019-08-13 21:45:30 +03:00
Translator.js Update eslint to v6 (#1777) 2019-08-13 21:45:30 +03:00
Translator.test.js Update eslint to v6 (#1777) 2019-08-13 21:45:30 +03:00