The `ErrorWithCause` class uses the same signature as the builtin Error
does, but it supports the `{ cause }` option in older environments, and
it propagates the `isNetworkError` property from the underlying error.
* @uppy/tus: pause all requests in response to server rate limiting
When the remote server responds with HTTP 429, all requests are
paused for a while in the hope that it can resolve the rate limiting.
Failed requests are also now queued up after the retry delay. Before
that, they were simply scheduled which would sometimes end up
overflowing the `limit` option.
* Address review comments
* fix requests bypassing queue pause state
* Auto rate limiting
* fix `RateLimitedQueue`
* rewrite to async/await
* Only fetch size (HEAD) if needed #3034
* Update packages/@uppy/companion/src/server/controllers/url.js
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* Change HEAD to GET in getURLMeta
and abort request immediately upon response headers received
https://github.com/transloadit/uppy/issues/3034#issuecomment-908059234
* fix lint
* fix lint
* cut off length of file names
or else we get
"MetadataTooLarge: Your metadata headers exceed the maximum allowed metadata size" in tus / S3
* try to fix flaky test
* remove iife and cleanup code a bit
* fix lint by reordering code
* rename Uploader to MultipartUploader
* Rewrite Uploader to use fs-capacitor #3098
This allows for upload to start almost immediately without having to first download the file.
And it allows for uploading bigger files, because transloadit assembly will not timeout,
as it will get upload progress events all the time.
No longer need for illusive progress.
Also fix eslint warnings and simplify logic
Still TODO: TUS pause/resume has a bug:
https://github.com/tus/tus-js-client/issues/275
* add comment in dev Dashboard and pull out variable
* fix a bug where remote xhr upload would ignore progress events in the UI
* fix bug where s3 multipart cancel wasn't working
* fix also cancel for xhr
* Rewrite providers to use streams
This removes the need for disk space as data will be buffered in memory and backpressure will be respected
https://github.com/transloadit/uppy/issues/3098#issuecomment-907763809
All providers "download" methods will now return a { stream } which can be consumed by uploader.
Also:
- Remove capacitor (no longer needed)
- Change Provider/SearchProvider API to async (Breaking change for custom companion providers)
- Fix the case with unknown length streams (zoom / google drive). Need to be downloaded first
- rewrite controllers deauth-callback, thumbnail, list, logout to async
- getURLMeta: make sure size is never NaN (NaN gets converted to null in JSON.stringify when sent to client but not when used in backend)
- fix purest mock (it wasn't returning statusCode on('response'))
- add missing http mock for "request" for THUMBNAIL_URL and http://url.myendpoint.com/file (these request errors were never caught by tests previously)
- "upload functions with tus protocol" test: move filename checking to new test where size is null. Fix broken expects
- fix some lint
* Implement streamingUpload flag
COMPANION_STREAMING_UPLOAD
Default to false due to backward compatibility
If set to true, will start to upload files at the same time as dowlnoading them, by piping the streams
- Also implement progress for downloading too
- and fix progress duplication logic
- fix test that assumed file was fully downloaded after first progress event
* rearrange validation logic
* add COMPANION_STREAMING_UPLOAD to env.test.sh too
* implement maxFileSize option in companion
for both unknown length and known length downloads
* fix bug
* fix memory leak when non 200 status
streams were being kept
* fix lint
* Add backward-compatibility for companion providers
Implement a new static field "version" on providers, which when not set to 2,
will cause a compatibility layer to be added for supporting old callback style provider api
also fix some eslint and rename some vars
* document new provider API
* remove static as it doesn't work on node 10
* try to fix build issue
* degrade to node 14 in github actions
due to hitting this error: https://github.com/nodejs/node/issues/40030https://github.com/transloadit/uppy/pull/3159/checks?check_run_id=3544858518
* pull out duplicated logic into reusable function
* fix lint
* make methods private
* re-add unsplash download_location request
got lost in merge
* add try/catch
as suggested https://github.com/transloadit/uppy/pull/3159#discussion_r727149263
* Only set default chunkSize if needed
for being more compliant with previous behavior when streamingUpload = false
* Improve flaky test
Trying to fix this error:
FAIL packages/@uppy/utils/src/delay.test.js
● delay › should reject when signal is aborted
expect(received).toBeLessThan(expected)
Expected: < 70
Received: 107
32 | const time = Date.now() - start
33 | expect(time).toBeGreaterThanOrEqual(30)
> 34 | expect(time).toBeLessThan(70)
| ^
35 | })
36 | })
37 |
at Object.<anonymous> (packages/@uppy/utils/src/delay.test.js:34:18)
https://github.com/transloadit/uppy/runs/3984613454?check_suite_focus=true
* Apply suggestions from code review
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* fix review feedback & lint
* Apply suggestions from code review
Co-authored-by: Merlijn Vos <merlijn@soverin.net>
* remove unneeded ts-ignore
* Update packages/@uppy/companion/src/server/controllers/url.js
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* Update packages/@uppy/companion/src/server/Uploader.js
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* reduce nesting
* fix lint
* optimize promisify
https://github.com/transloadit/uppy/pull/3159#discussion_r738160576
* Update packages/@uppy/companion/test/__tests__/uploader.js
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
Co-authored-by: Merlijn Vos <merlijn@soverin.net>
* rewrite to async/await
* Only fetch size (HEAD) if needed #3034
* Update packages/@uppy/companion/src/server/controllers/url.js
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* Change HEAD to GET in getURLMeta
and abort request immediately upon response headers received
https://github.com/transloadit/uppy/issues/3034#issuecomment-908059234
* fix lint
* fix lint
* cut off length of file names
or else we get
"MetadataTooLarge: Your metadata headers exceed the maximum allowed metadata size" in tus / S3
* try to fix flaky test
* remove iife and cleanup code a bit
* fix lint by reordering code
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* fix: AbortController is not defined
* use globalThis
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
* add stylelint with stylelint-config-standard
* autofix style issues
* fix more linting issues
* add stylelint-scss
* fix two more
* add stylelint-config-rational-order and autofix order
https://github.com/constverum/stylelint-config-rational-order
We can now use `Object.fromEntries`, `Object.values`, and `Symbol`.
Also changes polyfill suggestion for legacy browsers to use `core-js`
which includes almost all we need and more.
* Remove IE polyfills and special casing
* dashboard: remove `resize-observer-polyfill`
* deps: remove url-parse to use global `URL` instead
* Update documentation for browser support
* Add a migration guide
* Update e2e test platforms
* Remove polyfills from examples and add those missing in CDN bundle
* Use Node.js v16.x on CI because of `AbortController`
* Update bundle builder to produce two builds: `uppy.js` without polyfills, `uppy.legacy.js` with polyfills.
* Upgrade create-react-app deps
* remove null coalescing operator as it's not supported by the build chain
* disable Safari testing (for now)
Co-authored-by: Renée Kooi <renee@kooi.me>
* Translator: refactor interpolate
Fixes types and small code edits:
`<regex>[Symbol.split](<string>)` is equivalent to
`String.prototype.split.call(<string>, <regex>)` and potentially more
readable (the downside it's not compatible with IE, which may be why we
currently use the latter).
Refs: https://github.com/transloadit/uppy/pull/2899
* Update packages/@uppy/utils/src/Translator.js
* Revert RegExp Symbol for IE compat
* move findIndex to @uppy/utils, add tests
* use Array.findIndex instead of Array.indexOf in uppy.removePlugin
because strict comparison started failing with Vue3 Proxies
* dashboard: fix modal position on rtl pages, fixes#2495
* Align text to the right on right-to-left pages
* core: fix default text alignment on right-to-left pages
* dashboard: fix margin in provider browser on right-to-left pages
* dashboard: use direction-aware offset properties
* provider-views: use logical offset CSS properties
* status-bar: use logical offset CSS properties
* dashboard: fix addMore icon margin on RTL pages
* dashboard: some more logical properties
* build: compile logical properties to old CSS
* dashboard: set dir attribute if page did not provide one
* status-bar: set fallback text direction if not provided
* support direction option for inline and modal dashboard
* use companion.uppy.io on deploy previews
* provider-views: fix RTL alignment of file icons
* put the dir attribute on a wrapper div
* fix icon alignment in RTL provider views
* informer: fix ? alignment in RTL mode
* add passesRestrictions(file) and utilize it in provider-views
* disallow selecting more than maxNumberOfFiles
* add restriction reason and include folders
* let’s leave folder out of this for now — otherwise not only you can’t select them, but navigate too
* disallow selecting if totalCurrentSelectionFileSize is over maxTotalFileSize
* handle maxNumberOfFiles not being set 🙈
* fix totalCurrentSelectionFileSize and handle no file.name
* support grid in restrictions
* refactor the restriction reason logic, rename to validateRestrictions, wrap icon in a container
* add uppyFileSizes 🙈
* get rid of canSelectMore in favor of validateRestrictions (file, files)
* handle shift key bulk selection
* remove unused props
* use infoTimeout
I added support to load thumbnails when uploading AVIF images, by adding 'avif' to the regex. I also updated the unit test.
Note that support for this is dependant on browser support, and currently only Chrome 85+ and Firefox 77+ support it (Firefox requires flag to be set to allow it)
* Move @uppy/utils/lib/prettyBytes -> @transloadit/prettier-bytes so we can more easily re-use it across projects
* Switch to MIT licensed @transloadit/prettier-bytes
* lockfile fun
* build: add dep for website/inject.js
Co-authored-by: Renée Kooi <renee@kooi.me>
* Add NetworkError error type and isNetworkError utility check
* Detect network errors in tus and xhr-upload
Co-Authored-By: Ifedapo .A. Olarewaju <ifedapoolarewaju@gmail.com>
* add NetworkError check to RequestClient in companion-client
* return false if !xhr 🙈
* move catch so that only errors from _checkMinNumberOfFiles are caught — errors from uploads are logged in `upload-error` already
//cc @goto-bus-stop
* Add NetworkError to MiniXHRUpload in aws-s3
//cc @goto-bus-stop
* Add tests
//cc @ifedapoolarewaju
* pass xhr to NetworkError
* add logging in catch of uppy.upload()
* start error message with “This looks like a network error”
* originalRequest --> request, return NetworkError right away
* check for NetworkError in `delete` as well
do we need it in preflight too, @ifedapoolarewaju?
* remove redundant error message re-declaration
* _showOrLogErrorAndThrow, but don’t showInformer
Co-authored-by: Ifedapo .A. Olarewaju <ifedapoolarewaju@gmail.com>
* aws-s3: handle actual upload internally instead of deferring to XHRUpload
* aws-s3: add a comment explaining the mess
* aws-s3: move XHRUpload code into a separate class
* utils: support prioritization in rate limiting queue
* aws-s3: remove old implementation
* core,status-bar: account for pre/postprocessing files in progress calculations
* changelog: add entry for this PR
* utils: Support cancellation in `RateLimitedQueue#wrapPromiseFunction`
* s3: remove preprocessing stage, only use upload-* events
* utils: remove console.log
* aws-s3: comment out duplicate `upload-started`
* xhr-upload: reorder `ProgressTimeout` creation for readability
* aws-s3: apply 7d972e0 to MiniXHRUpload
* aws-s3: apply 4ed7508 to MiniXHRUpload
* webcam: request a video mime type listed in `allowedFileTypes`
* webcam: stop recording when we are about to reach the max file size
* webcam: request an image mime type listed in `allowedFileTypes`
* docs: add Webcam#preferredImageMimeType
* locale: add `recordingStoppedMaxSize` string
* changelog: check webcam restrictions
Co-authored-by: Artur Paikin <artur@arturpaikin.com>
