- add support for remote uploads
- add support for restore using golden-retriever
- update examples
- #6181 still persists
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Mikael Finstad <finstaden@gmail.com>
right now whenever you upload any file through multipart upload you'll
find empty body returned in `upload-success` event
cause : response from completeMultipartRequest returned in CamelCase
which was not parsed correctly
```xml
<?xml version="1.0" encoding="UTF-8"?>
<CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Location>https://testbucketnewfix.s3.eu-north-1.amazonaws.com/13febe01-8d5b-4ed3-bda1-9620fc37c694-VSCode-darwin-universal.dmg</Location><Bucket>testbucketnewfix</Bucket><Key>13febe01-8d5b-4ed3-bda1-9620fc37c694-VSCode-darwin-universal.dmg</Key><ETag>"be4129821e06a893b453f1373995202b-49"</ETag><ChecksumCRC64NVME>Wjr7mcJOfU0=</ChecksumCRC64NVME><ChecksumType>FULL_OBJECT</ChecksumType></CompleteMultipartUploadResult>
```
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
shouldUseMultipart: true was not working due to signature mismatch
AI summary :
This pull request improves support for specifying the `contentType`
during S3 uploads, ensuring that the correct content type is set when
generating presigned URLs for both single and multipart uploads. It also
updates the client-side and server-side code to handle this new
parameter.
**Backend and API enhancements:**
- Updated the `/s3/presign` endpoint in `index.js` to accept an optional
`contentType` parameter and to include it in the S3 `PutObjectCommand`
and `CreateMultipartUploadCommand` requests, defaulting to
`'application/octet-stream'` if not provided.
[[1]](diffhunk://#diff-0c4471088c62917198e1438777b4c7e78fd4d87d450d2a7c78b86b7c647ac97bL360-R360)
[[2]](diffhunk://#diff-0c4471088c62917198e1438777b4c7e78fd4d87d450d2a7c78b86b7c647ac97bR384-R391)
**Client-side and types improvements:**
- Added `contentType` to the `presignableRequest` type to ensure type
safety and clarity when passing this parameter.
- Modified the S3 client (`S3.ts`) to include `contentType` when making
presign requests.
- Updated the example HTML (`rewrite-test.html`) to send `contentType`
in presign requests and enabled multipart uploads for testing.
[[1]](diffhunk://#diff-ca9bc28a71d2d8ac4e0aa6844698f2244594a12e695fc3e153db36674ccb2b52L103-R103)
[[2]](diffhunk://#diff-ca9bc28a71d2d8ac4e0aa6844698f2244594a12e695fc3e153db36674ccb2b52R114)
Cherry-picked as the previous branch got messy ( from last PR )
- XHR-based uploads using `@uppy/utils/fetcher`
- Silent offline detection (without treating it as an error) with
automatic resume ( inspired from old plugin code )
- Retry with backoff using fetcher defaults:
- Max retries: 3
- Delays: 300ms → 600ms → 1200ms
- Retries on 5xx and 429; skips other 4xx errors
- S3 Error Mappings
Working :
- Simple uploads (putObject)
- Multipart uploads
- Pause / Resume / Abort , Resume works using listParts as before
- Progress tracking per chunk ( `#onProgress` ) - this is temporary ,
until we implement xhr based progress tracking as - before.
**added new examples to examples/aws-nodejs to test the rewrite , old
examples will eventually be removed**
Not Yet Implemented
- Rate limiting – Currently uploads are sequential
- Retries
- Remote file uploads – Files from Google Drive, Dropbox, etc.
XHR progress
- full wiring with uppy state ( I saw few state bugs while testing it
manually )
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Murderlon <merlijn@soverin.net>
#### This PR replaces signed headers with presigned urls , for signing.
- I implemented this along with the uploader rewrite , but raised PR for
this it separately to reduce diffs.
- `signer.ts` now returns signed url , which is directly used by
`_presignedRequest` method to make calls to s3
- tested it with uploader rewrite working fine.
---------
Co-authored-by: Murderlon <merlijn@soverin.net>
- added `getCredentials` callback in `S3Config`
- simplified signer test util ( `sigv4-signer.js` ) and moved to
`src/signer.ts`
- all tests run in a new non root minio user ( required for sts ) ,
which gets created before the tests start see `tests/s3-client/setup.js`
- tests added for sts , using `@aws-sdk/client-sts` for getting creds.
- credential caching inspired from the current implementation in
`@aws-s3` plugin , see `#getTemporarySecurityCredentials` inside
`@uppy/aws-s3/src/index.ts`
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
AI disclaimer : AI used
## Changes
- **jsdom 26 → 29** across all packages' devDependencies —
**except`@uppy/aws-s3`** (left at 26; that plugin is being rewritten and
its deps are handled on the rewrite branch).
- **`@uppy/xhr-upload` unit tests migrated from nock to MSW**, and
`nock` dropped from its devDependencies.
## Why the test migration was needed
jsdom **28** overhauled resource loading so `XMLHttpRequest` now goes
through jsdom's internal (undici-based) fetch instead of Node's `http`
module ([jsdom 28 release
notes](https://github.com/jsdom/jsdom/releases/tag/28.0.0)).
nock intercepts at the Node `http` / `http.ClientRequest` layer ([nock
README](https://github.com/nock/nock)), so it can no longer see uppy's
upload requests — the mocked responses never arrive and the tests time
out. (This is the long-standing nock + jsdom XHR incompatibility:
[nock#518](https://github.com/nock/nock/issues/518).)
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Part of the dev-dependency major-version upgrades. These required code
changes so they're split out from the no-code-change batch (#6315).
**AI Disclaimer** : AI Used
- **glob 8 → 13** — `@uppy/locales` scripts: the callback API was
removed, so `getPaths()` uses the promise form and `test.mjs` uses the
named `globSync`.
- **npm-packlist 5 → 11** — `upload-to-cdn.js`: v7+ takes an
`@npmcli/arborist` tree instead of `{ path }`; loads the tree and passes
it to `packlist()`. Adds `@npmcli/arborist` as a devDependency.
- **nock 13 → 14** (`@uppy/companion`) — the v14 @mswjs/interceptors
rewrite crashed the SSRF tests in `http-agent.test.ts`; nock
interception is now kept off by default and only activated for the one
test that mocks a host.
- **@types/use-sync-external-store 0 → 1** — `@uppy/react`: v1's exports
map only exposes extensionless subpaths, so imports use
`use-sync-external-store/shim` and `/with-selector` (runtime supports
both).
will do a separte PR for changes which would require code changes so
it's easier to review , didn't upgrade any deps for `@uppy/aws-s3` since
we'll merge the rewrite so I think we should do that upgrade in the
rewrite branch before merging.
update : Dependency upgrades which required code changes were raised
separately
- #6309
- #6310
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
this fixes#6287 , another similar attempted fix which didn't got merged
: #5683
in #5683 we discussed a more longer term solution but IMO that would be
a breaking change as it would change the event contract
AI Disclaimer : AI Used
This fix is Manually Tested across **Chrome and Firefox**
all the dev deps upgraded excepts for these :
- `@types/cors: 2.8.6`
- `@types/jsonwebtoken: 8.3.7`
- `@types/morgan: 1.7.37`
- `@types/node: ^20.19.0`
- `nock: ^13.1.3`
upgrading these to their latest major either breaks the test suite or
requires us to do code changes on our end.
#6297
---------
Co-authored-by: Mikael Finstad <finstaden@gmail.com>
exact copy of #6175
- I will point the uppy deps upgrade pr to this PR's branch
- no other changes needed in this branch as #6175 was reviewed and all
the comments were resolved
this should be the order of merge :
1. #6300
2. uppy deps upgrade ( yet to be raised )
3. merge this to main
4. merge biome changes #6244
@mifi let me know if you think otherwise.
---------
Co-authored-by: Mikael Finstad <finstaden@gmail.com>
Supersedes #6178.
This branch keeps the exact same final content as #6178, but splits
history for reviewability:
1. `chore(companion): rename source and test files from .js to .ts`
(pure `git mv`, no content changes)
2. `feat(companion): port Companion to TypeScript` (actual
content/type/schema/build updates)
Validation note:
- Final tree is byte-for-byte identical to `ts-companion`
(`7b5b16a298690b0492de4cc4fab744f998b45570`).
---------
Co-authored-by: Mikael Finstad <finstaden@gmail.com>
fixes#6181
`uploadRemoteFile()` now treats one remote upload attempt as a single
queue-owned unit. Instead of queueing “request token” and “open
websocket / wait for upload” as two separate jobs, it now:
- acquires one queue slot
- reuses serverToken if resuming, otherwise requests a new
token
- immediately proceeds to the websocket/upload phase within
that same admitted attempt
That removes the stale-token race where Companion starts its 60s
socket-wait timer before the client’s websocket phase has actually been
admitted by the queue. Retries still reacquire the queue per attempt,
and existing abort/cancel behavior is
preserved.
refer
https://github.com/transloadit/uppy/issues/6181#issuecomment-4295036192
for more context about the bug
Fixes#6256
## Problem
When `fileManagerSelectionType` is set to `'folders'`, clicking the **My
Device** tab/button
still opens the file picker instead of the folder picker.
The inline "browse" link in the tagline correctly respects the setting,
but
`renderMyDeviceAcquirer` in `AddFiles.tsx` always called
`triggerFileInputClick`
regardless of the `fileManagerSelectionType` prop.
## Solution
`renderMyDeviceAcquirer` now reads `fileManagerSelectionType` and calls
`triggerFolderInputClick` when set to `'folders'`, or
`triggerFileInputClick`
for `'files'` and `'both'`.
For `'both'` mode, the button defaults to file selection because a
single
HTML `<input>` cannot handle both files and folders simultaneously
(`webkitdirectory` is all-or-nothing). The folder picker remains
accessible
via the tagline's "browse folders" link.
---------
Co-authored-by: Prakash <qxprakash@gmail.com>
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and the packages will
be published to npm automatically. If you're not ready to do a release
yet, that's fine, whenever you add more changesets to main, this PR will
be updated.
# Releases
## @uppy/utils@7.2.0
### Minor Changes
- 6b1abaa: Introduce modern, minimal TaskQueue to replace
RateLimitedQueue
## @uppy/xhr-upload@5.2.0
### Minor Changes
- 6b1abaa: Introduce modern, minimal TaskQueue to replace
RateLimitedQueue
### Patch Changes
- Updated dependencies [6b1abaa]
- @uppy/utils@7.2.0
## @uppy/companion@6.2.2
### Patch Changes
- 49db42d: Fix bug with 429 not returning JSON response with message
- 4652dc6: upgrade @aws-sdk/ deps in @uppy/companion
## @uppy/transloadit@5.5.1
### Patch Changes
- 4787960: Add type re-export for `AssemblyInstructionsInput`
- 2f6849d: Fix allowMultipleUploadBatches to prevent adding/removing
files while an upload is in progress (#6156)
- 29d2772: remove monkey patch from uppy bundle package
- Updated dependencies [6b1abaa]
- @uppy/utils@7.2.0
## uppy@5.2.4
### Patch Changes
- 29d2772: remove monkey patch from uppy bundle package
- a86c624: upgrade esbuild in uppy to "^0.27.0"
- Updated dependencies [6b1abaa]
- Updated dependencies [4787960]
- Updated dependencies [2f6849d]
- Updated dependencies [29d2772]
- @uppy/xhr-upload@5.2.0
- @uppy/transloadit@5.5.1
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
The `AssemblyInstructionsInput` type from `@transloadit/types` is now
publicly exported from `@uppy/transloadit`, allowing consumers to type
their assembly parameters without importing from the external package.
## Changes
- Re-exported `AssemblyInstructionsInput` type in
`packages/@uppy/transloadit/src/index.ts`
## Usage
```typescript
import type { AssemblyInstructionsInput } from '@uppy/transloadit'
const params: AssemblyInstructionsInput = {
steps: {
encode: {
robot: '/video/encode',
use: ':original',
preset: 'mp4',
},
},
}
```
The type was already imported internally and aliased as
`AssemblyParameters`. This change makes it available to plugin
consumers.
<!-- START COPILOT ORIGINAL PROMPT -->
<details>
<summary>Original prompt</summary>
> Make the AssemblyInstructionsInput type available from the
@uppy/transloadit package by re-exporting it from
packages/@uppy/transloadit/src/index.ts.
>
> Background
> - The Transloadit plugin imports the type AssemblyInstructionsInput
from the external package @transloadit/types and aliases it as
AssemblyParameters in packages/@uppy/transloadit/src/index.ts.
> - Consumers of the @uppy/transloadit package need direct access to the
AssemblyInstructionsInput type but it is not currently exported from the
plugin package.
>
> Change requested
> - Add a type re-export so that consumers can import
AssemblyInstructionsInput from the transloadit plugin package, e.g.
> import type { AssemblyInstructionsInput } from '@uppy/transloadit'
>
> Files to change
> - packages/@uppy/transloadit/src/index.ts
> - Add the following export alongside the other package exports (near
the end of the module where other exports live):
>
> ```typescript name=packages/@uppy/transloadit/src/index.ts
url=https://github.com/transloadit/uppy/blob/main/packages/@uppy/transloadit/src/index.ts
> -export { COMPANION_URL, COMPANION_ALLOWED_HOSTS }
> +export { COMPANION_URL, COMPANION_ALLOWED_HOSTS }
> +
> +// Re-export type from @transloadit/types so callers can import it
from the plugin package.
> +export type { AssemblyInstructionsInput } from '@transloadit/types'
> ```
>
> Notes
> - This is a purely type-only export and should be erased at compile
time; it does not add runtime code.
> - No other behavior or API surface should change.
>
> Please create a branch, apply the change, and open a pull request with
a descriptive title and a short description of the change. Run
TypeScript build checks if available.
>
</details>
<!-- START COPILOT CODING AGENT SUFFIX -->
*This pull request was created from Copilot chat.*
>
<!-- START COPILOT CODING AGENT TIPS -->
---
💬 We'd love your input! Share your thoughts on Copilot coding agent in
our [2 minute survey](https://gh.io/copilot-coding-agent-survey).
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mifi <402547+mifi@users.noreply.github.com>
Co-authored-by: Mikael Finstad <finstaden@gmail.com>
this fixes #6167
AI summary :
**The Bug: S3 Multipart Upload Fails with InvalidPart**
**What happened**
When uploading large files (200MB+) from Google Drive → Companion → S3
using multipart upload, the upload would reach ~96-100% and then fail
with S3's `InvalidPart` error on `CompleteMultipartUpload`.
*Root cause*
_A version mismatch between two AWS SDK packages caused by yarn dedupe._
Commit `3c3034b40` ("Dedupe dependencies #6085") ran `yarn dedupe`.
Companion had `@aws-sdk/client-s3` and `@aws-sdk/lib-storage` both at
`"^3.338.0"`, which previously both resolved to `3.600.0`. But the
`@uppy/transloadit` package elsewhere in the monorepo had
`"@aws-sdk/client-s3": "^3.891.0"`. Dedupe merged the resolutions —
`client-s3` jumped to `3.896.0` while `lib-storage` stayed at `3.600.0`.
*Why the mismatch matters*
AWS SDK `client-s3@3.896.0` introduced a _breaking behavioral change_ in
its checksum middleware:
• *Old (3.600.0):* Default checksum = MD5, no auto-injection.
`UploadPart` requests sent _no checksum header_.
• *New (3.896.0):* Default checksum = CRC32,
`requestChecksumCalculation` defaults to `"WHEN_SUPPORTED"`. The
middleware _auto-injects `x-amz-checksum-crc32`_ on every `UploadPart`
request.
The old `lib-storage@3.600.0` didn't know about this new behavior. It
called `CreateMultipartUpload` _without_ setting `ChecksumAlgorithm`.
Then `client-s3@3.896.0` added CRC32 checksums to each `UploadPart`. S3
saw parts with checksums that weren't declared at creation time →
rejected with `InvalidPart` at `CompleteMultipartUpload`.
*How the fixes work*
_Option A — `requestChecksumCalculation: 'WHEN_REQUIRED'`_ (workaround):
Tells the new `client-s3` to only add checksums when S3 requires them
(which it doesn't for `UploadPart`). This restores the old behavior — no
checksum headers, no mismatch.
_Option B — Upgrade all AWS SDK packages to `^3.986.0`_ (proper fix):
The new `lib-storage@3.986.0` is _aware_ of the CRC32 checksum behavior.
When it detects `requestChecksumCalculation = "WHEN_SUPPORTED"`, it
explicitly sets `ChecksumAlgorithm: "CRC32"` on the
`CreateMultipartUpload` call. Now S3 expects CRC32 checksums on the
parts, the parts have CRC32 checksums → everything is consistent →
upload succeeds.
*TL;DR*
`yarn dedupe` upgraded `client-s3` but not `lib-storage`. The new
`client-s3` started adding CRC32 checksums to parts, but the old
`lib-storage` didn't declare CRC32 at multipart creation time. S3
rejected the mismatch. Fix: upgrade both packages together so they agree
on checksum behavior.
## Implementation Plan for Transloadit allowMultipleUploadBatches Fix
- [x] Explore the codebase to understand existing structure
- [x] Review transloadit/src/index.ts
- [x] Review existing event handlers (#onError, #onCancelAll)
- [x] Review install() and uninstall() methods
- [x] Understand test structure
- [x] Implement the proper fix in
packages/@uppy/transloadit/src/index.ts
- [x] Set allowNewUpload: false at start of #prepareUpload
(preprocessor)
- [x] Reset allowNewUpload: true at end of #afterUpload (postprocessor)
- [x] Reset allowNewUpload: true in #prepareUpload error handler
- [x] Keep existing resets in #onError and #onCancelAll handlers
- [x] Removed event listener approach (was causing race conditions)
- [x] Update tests to match implementation
- [x] Test allowNewUpload lifecycle through actual upload flow
(preprocessor/postprocessor)
- [x] Test allowNewUpload reset on preprocessor error
- [x] Test allowNewUpload reset on error event
- [x] Test allowNewUpload reset on cancel-all
- [x] Remove obsolete event listener tests
<!-- START COPILOT CODING AGENT TIPS -->
---
💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mifi <402547+mifi@users.noreply.github.com>
Co-authored-by: Mikael Finstad <finstaden@gmail.com>
Closes#4173, kind off
## Problem
Current situation with `RateLimitedQueue`
- Track concurrency: keep a running count, only dispatch up to limit,
accept Infinity.
- Priority enqueue: queued handlers sorted by priority; dequeued in that
order.
- Lifecycle bookkeeping: each job gets abort()/done() hooks; abort for
running vs queued differs; decrement active count and advance queue via
microtasks.
- Requeue placeholders: supports a shouldBeRequeued marker so callers
can hold/retry slots without running immediately.
- Function wrappers: wrap sync or async fns to enforce the queue and
return abortable handles.
- Cancellation plumbing: provides abortOn(signal) to bind queued/running
work to an AbortSignal; abortable promises carry abort().
- Pausing: pause(duration?) freezes dispatch (optional auto-resume);
resume() restarts up to the current limit.
- Rate limiting/backoff: rateLimit(duration) pauses, drops concurrency,
then ramps it back toward the previous upper bound over time.
Case in point: it's some sort of made up, monstrosity data structure
trying to be too many things. It also has rate limiting and exponential
backoff but that's already in
[`fetcher`](https://github.com/transloadit/uppy/blob/main/packages/%40uppy/utils/src/fetcher.ts)
too.
Would be better if we had separation of concerns and proven data
structures.
## Solution
A "dumb" promise-based priority queue that doesn't care at all about
what a promise does or if it needs to be retried. The promise inside
determines if it has retrying and exponential backoff, such as a
`fetcher` promise.
To not make this a breaking change and a huge diff, we still implement
this per uploader, starting with xhr-upload, and keep backwards
compatibility in the interface so we can still pass it to
`companion-client`, which needs to share the same queue.
## Ideal future
When you think about it, it's odd that we implement a queue per uploader
if a queue is so central to uppy working correctly. It's even more odd
that we also have to inject that queue into `companion-client` per
uploader for queue sharing.
Ideally, `core` is responsible for having the queue. All uploaders would
do is push a promise to `core` and `core` doesn't care if that promise
is a tus, xhr, or S3 upload.
---------
Co-authored-by: Prakash <qxprakash@gmail.com>
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and the packages will
be published to npm automatically. If you're not ready to do a release
yet, that's fine, whenever you add more changesets to main, this PR will
be updated.
# Releases
## @uppy/components@1.2.0
### Minor Changes
- 37f69d0: Introduce `useImageEditor` to build your own UI for using our
image editor in React, Vue, and Svelte.
### Patch Changes
- 37f69d0: - Fix Vue components to work with kebab-case props
(`:edit-file` instead of `:editFile`)
- Updated dependencies [37f69d0]
- @uppy/image-editor@4.2.0
## @uppy/image-editor@4.2.0
### Minor Changes
- 37f69d0: Introduce `useImageEditor` to build your own UI for using our
image editor in React, Vue, and Svelte.
## @uppy/react@5.2.0
### Minor Changes
- 37f69d0: Introduce `useImageEditor` to build your own UI for using our
image editor in React, Vue, and Svelte.
### Patch Changes
- Updated dependencies [37f69d0]
- Updated dependencies [37f69d0]
- @uppy/components@1.2.0
## @uppy/svelte@5.2.0
### Minor Changes
- 37f69d0: Introduce `useImageEditor` to build your own UI for using our
image editor in React, Vue, and Svelte.
### Patch Changes
- Updated dependencies [37f69d0]
- Updated dependencies [37f69d0]
- @uppy/components@1.2.0
## @uppy/transloadit@5.5.0
### Minor Changes
- 37f69d0: Migrate from 'transloadit' to '@transloadit/types' to get the
types. No need to drag in the entire SDK.
### Patch Changes
- Updated dependencies [37f69d0]
- @uppy/tus@5.1.1
## @uppy/vue@3.2.0
### Minor Changes
- 37f69d0: Introduce `useImageEditor` to build your own UI for using our
image editor in React, Vue, and Svelte.
### Patch Changes
- 37f69d0: - Fix Vue components to work with kebab-case props
(`:edit-file` instead of `:editFile`)
- Updated dependencies [37f69d0]
- Updated dependencies [37f69d0]
- @uppy/components@1.2.0
## @uppy/locales@5.1.1
### Patch Changes
- 37f69d0: Update cs_CZ dropPaste keys to use the correct variables.
## @uppy/tus@5.1.1
### Patch Changes
- 37f69d0: Fix Node.js support by conditionally setting a property which
does not exist in Node.js instead of crashing.
## uppy@5.2.3
### Patch Changes
- Updated dependencies [37f69d0]
- Updated dependencies [37f69d0]
- Updated dependencies [37f69d0]
- Updated dependencies [37f69d0]
- @uppy/locales@5.1.1
- @uppy/tus@5.1.1
- @uppy/transloadit@5.5.0
- @uppy/image-editor@4.2.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
