diff --git a/docs/src/data/join-x.csv b/docs/src/data/join-x.csv new file mode 100644 index 000000000..077ad7434 --- /dev/null +++ b/docs/src/data/join-x.csv @@ -0,0 +1,4 @@ +a,b,c +a,t,1 +b,u,2 +c,v,3 diff --git a/docs/src/data/join-y.csv b/docs/src/data/join-y.csv new file mode 100644 index 000000000..d3f7bb401 --- /dev/null +++ b/docs/src/data/join-y.csv @@ -0,0 +1,4 @@ +e,f,g +a,t,3 +b,u,2 +d,w,1 diff --git a/docs/src/questions-about-joins.md b/docs/src/questions-about-joins.md index d0a118462..d7f7e70e2 100644 --- a/docs/src/questions-about-joins.md +++ b/docs/src/questions-about-joins.md @@ -139,6 +139,84 @@ Thanks to @aborruso for the tip! See also the [record-heterogeneity page](record-heterogeneity.md). +## Doing SQL-style left, right, inner, and full-outer joins + +Miller's `join` verb is defined in terms of _paired_ and _unpaired_ records, rather than SQL-database terminology -- but you can get SQL-style joins using the `--ul` and `--ur` flags (which emit unpaired left-file and right-file records, respectively), along with [`unsparsify`](reference-verbs.md#unsparsify) to fill in empty cells for non-matches. + +Suppose you have the following two data files, where we want to join on the left file's `a` field matching the right file's `e` field: + +
+a,b,c +a,t,1 +b,u,2 +c,v,3 ++ +
+e,f,g +a,t,3 +b,u,2 +d,w,1 ++ +In all the following examples, the `-f` file (`data/join-x.csv`) is the left file, and the file in the main input stream (`data/join-y.csv`) is the right file. The flags `-j a -r e` say that the left file's `a` field is matched against the right file's `e` field, with the output join column named `a`. + +**Inner join** -- only matching records -- is what Miller's `join` does by default, since only paired records are emitted: + +
+mlr --icsv --ocsv join -j a -r e -f data/join-x.csv data/join-y.csv ++
+a,b,c,f,g +a,t,1,t,3 +b,u,2,u,2 ++ +**Left join** keeps all records from the left file, with empty cells where the right file has no match. Use `--ul` to also emit unpaired left-file records, then `unsparsify` to square up the output: + +
+mlr --icsv --ocsv join --ul -j a -r e -f data/join-x.csv \ + then unsparsify --fill-with "" \ + data/join-y.csv ++
+a,b,c,f,g +a,t,1,t,3 +b,u,2,u,2 +c,v,3,, ++ +**Right join** keeps all records from the right file. Use `--ur` to also emit unpaired right-file records: + +
+mlr --icsv --ocsv join --ur -j a -r e -f data/join-x.csv \ + then unsparsify --fill-with "" \ + data/join-y.csv ++
+a,b,c,f,g +a,t,1,t,3 +b,u,2,u,2 +d,,,w,1 ++ +**Full outer join** keeps all records from both files. Use both `--ul` and `--ur`: + +
+mlr --icsv --ocsv join --ul --ur -j a -r e -f data/join-x.csv \ + then unsparsify --fill-with "" \ + data/join-y.csv ++
+a,b,c,f,g +a,t,1,t,3 +b,u,2,u,2 +d,,,w,1 +c,v,3,, ++ +Note that unpaired records are emitted after all paired records, so the output ordering may differ from what a SQL database would produce; you can pipe the output through [`sort`](reference-verbs.md#sort) if you need a particular ordering. + ## Doing multiple joins Suppose we have the following data: diff --git a/docs/src/questions-about-joins.md.in b/docs/src/questions-about-joins.md.in index 0066c27cb..368e6bda5 100644 --- a/docs/src/questions-about-joins.md.in +++ b/docs/src/questions-about-joins.md.in @@ -60,6 +60,50 @@ Thanks to @aborruso for the tip! See also the [record-heterogeneity page](record-heterogeneity.md). +## Doing SQL-style left, right, inner, and full-outer joins + +Miller's `join` verb is defined in terms of _paired_ and _unpaired_ records, rather than SQL-database terminology -- but you can get SQL-style joins using the `--ul` and `--ur` flags (which emit unpaired left-file and right-file records, respectively), along with [`unsparsify`](reference-verbs.md#unsparsify) to fill in empty cells for non-matches. + +Suppose you have the following two data files, where we want to join on the left file's `a` field matching the right file's `e` field: + +GENMD-INCLUDE-ESCAPED(data/join-x.csv) + +GENMD-INCLUDE-ESCAPED(data/join-y.csv) + +In all the following examples, the `-f` file (`data/join-x.csv`) is the left file, and the file in the main input stream (`data/join-y.csv`) is the right file. The flags `-j a -r e` say that the left file's `a` field is matched against the right file's `e` field, with the output join column named `a`. + +**Inner join** -- only matching records -- is what Miller's `join` does by default, since only paired records are emitted: + +GENMD-RUN-COMMAND +mlr --icsv --ocsv join -j a -r e -f data/join-x.csv data/join-y.csv +GENMD-EOF + +**Left join** keeps all records from the left file, with empty cells where the right file has no match. Use `--ul` to also emit unpaired left-file records, then `unsparsify` to square up the output: + +GENMD-RUN-COMMAND +mlr --icsv --ocsv join --ul -j a -r e -f data/join-x.csv \ + then unsparsify --fill-with "" \ + data/join-y.csv +GENMD-EOF + +**Right join** keeps all records from the right file. Use `--ur` to also emit unpaired right-file records: + +GENMD-RUN-COMMAND +mlr --icsv --ocsv join --ur -j a -r e -f data/join-x.csv \ + then unsparsify --fill-with "" \ + data/join-y.csv +GENMD-EOF + +**Full outer join** keeps all records from both files. Use both `--ul` and `--ur`: + +GENMD-RUN-COMMAND +mlr --icsv --ocsv join --ul --ur -j a -r e -f data/join-x.csv \ + then unsparsify --fill-with "" \ + data/join-y.csv +GENMD-EOF + +Note that unpaired records are emitted after all paired records, so the output ordering may differ from what a SQL database would produce; you can pipe the output through [`sort`](reference-verbs.md#sort) if you need a particular ordering. + ## Doing multiple joins Suppose we have the following data: