* show how many files are added when loading
remake of https://github.com/transloadit/uppy/pull/4388
* add french (cherry pick)
* implement concurrent file listing
* refactor / fix lint
* refactor/reduce duplication
* pull out totals validation
don't do it for every file added, as it's very slow
instead do the check at the end when all files are added.
this allows us to easily work with 10k+ files
fixes#4389
* Update packages/@uppy/core/src/Uppy.js
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* make restricter.validate validate everything
instead make more specific methods for sub-validation
also rename validateTotals to validateAggregateRestrictions
* improve errors and user feedback
- handle errors centrally so that we can limit the amount of toasts (informers) sent to the users (prevent flooding hundreds/thousands of them)
- introduce FileRestrictionError which is a restriction error for a specific file
- introduce isUserFacing field for RestrictionError
* fix performance issue reintroduced
* improvements
- show "%{count} additional restrictions were not fulfilled" for any restriction errors more than 4
- refactor/rename methods
- improve ghost logic/comments
* improve performance when uploading
- introduce new event "upload-start" that can contain multiple files
- make a new patchFilesState method to allow updating more files
- unify "upload-start" logic in all plugins (send it before files start uploading)
- defer slicing buffer until we need the data
- refactor to reuse code
* fix e2e build issue
* try to upgrade cypress
maybe it fixes the error
* Revert "fix e2e build issue"
This reverts commit ff3e580c0f.
* upgrade parcel
* move mutation logic to end
* remove FileRestrictionError
merge it with RestrictionError
* fix silly bug
looks like the e2e tests are doing its job 👏
---------
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* fix race condtion when adding files
don't call addFolder from listAllFiles
because that would call addFile before all folders were loaded
also remove unused selectedFolders state
* fix also the case of adding multiple folders
* fix todo: remove SharedHandler
* remove loaderWrapper
* fix logic
* fix the last race condition
* fix broken duplicate file check
* fix logic
* prettiyfy loop
* add user feedback
so they know that something is happening
* run corepack yarn run build:locale-pack
* Revert "run corepack yarn run build:locale-pack"
This reverts commit 85548ce2fc.
* Revert "add user feedback"
This reverts commit 4060019c35.
* use async generator instead of p-map
* re-fix race-condition
* remove providerFileToId
as suggested by @arturi
* use addFiles instead of addFile
* rename function
* use provider-supplied file ID
instead of generating an ID, for providers that we whitelsit
this allows adding the same time many times (with a different path)
* call core directly
* improve dev dashboard
* disable experimental getAsFileSystemHandle
it seems to be causing problems when dropping folders with subfolders in newest chrome
e.g a folder with 50 files and a subfolder which in turn has another 50 files
also refactor and document the code more
* Update packages/@uppy/provider-views/src/ProviderView/ProviderView.jsx
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* mov eto utils
---------
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* workaround chrome crash
fixes#4133
* Update packages/@uppy/utils/src/getDroppedFiles/utils/webkitGetAsEntryApi/index.js
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
---------
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
`webkitGetAsEntry` is a non-standard/deprecated API, replacing it with
`getAsFileSystemHandle` when available.
This also work around a Chromium bug with symlinks.
Fixes: https://github.com/transloadit/uppy/issues/3505.
The `ErrorWithCause` class uses the same signature as the builtin Error
does, but it supports the `{ cause }` option in older environments, and
it propagates the `isNetworkError` property from the underlying error.
* @uppy/tus: pause all requests in response to server rate limiting
When the remote server responds with HTTP 429, all requests are
paused for a while in the hope that it can resolve the rate limiting.
Failed requests are also now queued up after the retry delay. Before
that, they were simply scheduled which would sometimes end up
overflowing the `limit` option.
* Address review comments
* fix requests bypassing queue pause state
* Auto rate limiting
* fix `RateLimitedQueue`
* rewrite to async/await
* Only fetch size (HEAD) if needed #3034
* Update packages/@uppy/companion/src/server/controllers/url.js
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* Change HEAD to GET in getURLMeta
and abort request immediately upon response headers received
https://github.com/transloadit/uppy/issues/3034#issuecomment-908059234
* fix lint
* fix lint
* cut off length of file names
or else we get
"MetadataTooLarge: Your metadata headers exceed the maximum allowed metadata size" in tus / S3
* try to fix flaky test
* remove iife and cleanup code a bit
* fix lint by reordering code
* rename Uploader to MultipartUploader
* Rewrite Uploader to use fs-capacitor #3098
This allows for upload to start almost immediately without having to first download the file.
And it allows for uploading bigger files, because transloadit assembly will not timeout,
as it will get upload progress events all the time.
No longer need for illusive progress.
Also fix eslint warnings and simplify logic
Still TODO: TUS pause/resume has a bug:
https://github.com/tus/tus-js-client/issues/275
* add comment in dev Dashboard and pull out variable
* fix a bug where remote xhr upload would ignore progress events in the UI
* fix bug where s3 multipart cancel wasn't working
* fix also cancel for xhr
* Rewrite providers to use streams
This removes the need for disk space as data will be buffered in memory and backpressure will be respected
https://github.com/transloadit/uppy/issues/3098#issuecomment-907763809
All providers "download" methods will now return a { stream } which can be consumed by uploader.
Also:
- Remove capacitor (no longer needed)
- Change Provider/SearchProvider API to async (Breaking change for custom companion providers)
- Fix the case with unknown length streams (zoom / google drive). Need to be downloaded first
- rewrite controllers deauth-callback, thumbnail, list, logout to async
- getURLMeta: make sure size is never NaN (NaN gets converted to null in JSON.stringify when sent to client but not when used in backend)
- fix purest mock (it wasn't returning statusCode on('response'))
- add missing http mock for "request" for THUMBNAIL_URL and http://url.myendpoint.com/file (these request errors were never caught by tests previously)
- "upload functions with tus protocol" test: move filename checking to new test where size is null. Fix broken expects
- fix some lint
* Implement streamingUpload flag
COMPANION_STREAMING_UPLOAD
Default to false due to backward compatibility
If set to true, will start to upload files at the same time as dowlnoading them, by piping the streams
- Also implement progress for downloading too
- and fix progress duplication logic
- fix test that assumed file was fully downloaded after first progress event
* rearrange validation logic
* add COMPANION_STREAMING_UPLOAD to env.test.sh too
* implement maxFileSize option in companion
for both unknown length and known length downloads
* fix bug
* fix memory leak when non 200 status
streams were being kept
* fix lint
* Add backward-compatibility for companion providers
Implement a new static field "version" on providers, which when not set to 2,
will cause a compatibility layer to be added for supporting old callback style provider api
also fix some eslint and rename some vars
* document new provider API
* remove static as it doesn't work on node 10
* try to fix build issue
* degrade to node 14 in github actions
due to hitting this error: https://github.com/nodejs/node/issues/40030https://github.com/transloadit/uppy/pull/3159/checks?check_run_id=3544858518
* pull out duplicated logic into reusable function
* fix lint
* make methods private
* re-add unsplash download_location request
got lost in merge
* add try/catch
as suggested https://github.com/transloadit/uppy/pull/3159#discussion_r727149263
* Only set default chunkSize if needed
for being more compliant with previous behavior when streamingUpload = false
* Improve flaky test
Trying to fix this error:
FAIL packages/@uppy/utils/src/delay.test.js
● delay › should reject when signal is aborted
expect(received).toBeLessThan(expected)
Expected: < 70
Received: 107
32 | const time = Date.now() - start
33 | expect(time).toBeGreaterThanOrEqual(30)
> 34 | expect(time).toBeLessThan(70)
| ^
35 | })
36 | })
37 |
at Object.<anonymous> (packages/@uppy/utils/src/delay.test.js:34:18)
https://github.com/transloadit/uppy/runs/3984613454?check_suite_focus=true
* Apply suggestions from code review
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* fix review feedback & lint
* Apply suggestions from code review
Co-authored-by: Merlijn Vos <merlijn@soverin.net>
* remove unneeded ts-ignore
* Update packages/@uppy/companion/src/server/controllers/url.js
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* Update packages/@uppy/companion/src/server/Uploader.js
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* reduce nesting
* fix lint
* optimize promisify
https://github.com/transloadit/uppy/pull/3159#discussion_r738160576
* Update packages/@uppy/companion/test/__tests__/uploader.js
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
Co-authored-by: Merlijn Vos <merlijn@soverin.net>
* rewrite to async/await
* Only fetch size (HEAD) if needed #3034
* Update packages/@uppy/companion/src/server/controllers/url.js
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* Change HEAD to GET in getURLMeta
and abort request immediately upon response headers received
https://github.com/transloadit/uppy/issues/3034#issuecomment-908059234
* fix lint
* fix lint
* cut off length of file names
or else we get
"MetadataTooLarge: Your metadata headers exceed the maximum allowed metadata size" in tus / S3
* try to fix flaky test
* remove iife and cleanup code a bit
* fix lint by reordering code
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* fix: AbortController is not defined
* use globalThis
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* add stylelint with stylelint-config-standard
* autofix style issues
* fix more linting issues
* add stylelint-scss
* fix two more
* add stylelint-config-rational-order and autofix order
https://github.com/constverum/stylelint-config-rational-order
We can now use `Object.fromEntries`, `Object.values`, and `Symbol`.
Also changes polyfill suggestion for legacy browsers to use `core-js`
which includes almost all we need and more.
* Remove IE polyfills and special casing
* dashboard: remove `resize-observer-polyfill`
* deps: remove url-parse to use global `URL` instead
* Update documentation for browser support
* Add a migration guide
* Update e2e test platforms
* Remove polyfills from examples and add those missing in CDN bundle
* Use Node.js v16.x on CI because of `AbortController`
* Update bundle builder to produce two builds: `uppy.js` without polyfills, `uppy.legacy.js` with polyfills.
* Upgrade create-react-app deps
* remove null coalescing operator as it's not supported by the build chain
* disable Safari testing (for now)
Co-authored-by: Renée Kooi <renee@kooi.me>
* Translator: refactor interpolate
Fixes types and small code edits:
`<regex>[Symbol.split](<string>)` is equivalent to
`String.prototype.split.call(<string>, <regex>)` and potentially more
readable (the downside it's not compatible with IE, which may be why we
currently use the latter).
Refs: https://github.com/transloadit/uppy/pull/2899
* Update packages/@uppy/utils/src/Translator.js
* Revert RegExp Symbol for IE compat
* move findIndex to @uppy/utils, add tests
* use Array.findIndex instead of Array.indexOf in uppy.removePlugin
because strict comparison started failing with Vue3 Proxies
* dashboard: fix modal position on rtl pages, fixes#2495
* Align text to the right on right-to-left pages
* core: fix default text alignment on right-to-left pages
* dashboard: fix margin in provider browser on right-to-left pages
* dashboard: use direction-aware offset properties
* provider-views: use logical offset CSS properties
* status-bar: use logical offset CSS properties
* dashboard: fix addMore icon margin on RTL pages
* dashboard: some more logical properties
* build: compile logical properties to old CSS
* dashboard: set dir attribute if page did not provide one
* status-bar: set fallback text direction if not provided
* support direction option for inline and modal dashboard
* use companion.uppy.io on deploy previews
* provider-views: fix RTL alignment of file icons
* put the dir attribute on a wrapper div
* fix icon alignment in RTL provider views
* informer: fix ? alignment in RTL mode
* add passesRestrictions(file) and utilize it in provider-views
* disallow selecting more than maxNumberOfFiles
* add restriction reason and include folders
* let’s leave folder out of this for now — otherwise not only you can’t select them, but navigate too
* disallow selecting if totalCurrentSelectionFileSize is over maxTotalFileSize
* handle maxNumberOfFiles not being set 🙈
* fix totalCurrentSelectionFileSize and handle no file.name
* support grid in restrictions
* refactor the restriction reason logic, rename to validateRestrictions, wrap icon in a container
* add uppyFileSizes 🙈
* get rid of canSelectMore in favor of validateRestrictions (file, files)
* handle shift key bulk selection
* remove unused props
* use infoTimeout
