Sample CSV data file:
+Suppose you have this CSV data file:
mlr cat is like cat ...
mlr cat is like cat -- it passes the data
+through unmodified ...
... but it can also do format conversion (here, to pretty-printed tabular format):
+... but it can also do format conversion (here, you can pretty-print in tabular format):
mlr head and mlr tail count records rather than lines. The CSV
-header is included either way:
mlr head and mlr tail count
+records rather than lines. Whethere you’re getting the first few records
+or the last few, the CSV header is included either way:
Sort primarily alphabetically on one field, then secondarily -numerically descending on another field:
+You can sort primarily alphabetically on one field, then +secondarily numerically descending on another field:
Use cut to retain only specified fields, in input-data order:
You can use cut to retain only specified fields, in the
+same order they appeared in the input data:
Use cut -o to retain only specified fields, in your specified order:
You can also use cut -o to retain only
+specified fields in your preferred order:
Use cut -x to omit specified fields:
You can use cut -x to omit fields you
+don’t care about:
Use filter to retain specified records:
You can use filter to keep only records you care about:
Use put to add/replace fields which are computed from other fields:
You can use put to create new fields which
+are computed from other fields:
JSON output:
+OK, CSV and pretty-print are fine. But Miller can also convert between +a few other formats -- let’s take a look at JSON output:
JSON output with vertical-formatting flags:
+Or, JSON output with vertical-formatting flags:
-$ mlr --icsv --ojson --jvstack --jlistwrap tail -n 2 example.csv
-[
+$ mlr --icsv --ojsonx tail -n 2 example.csv
{
"color": "yellow",
"shape": "circle",
@@ -380,7 +392,7 @@ $ mlr --icsv --ojson --jvstack --jlistwrap tail -n 2 example.csv
"quantity": 63.5058,
"rate": 8.3350
}
-,{
+{
"color": "purple",
"shape": "square",
"flag": 0,
@@ -388,16 +400,37 @@ $ mlr --icsv --ojson --jvstack --jlistwrap tail -n 2 example.csv
"quantity": 72.3735,
"rate": 8.2430
}
-]
Use then to pipe commands together. Also, the
--g option for many Miller commands is for group-by: here, head -n
-1 -g shape outputs the first record for each distinct value of the
-shape field. This means we’re finding the record with highest
-index field for each distinct shape field:
Now suppose you want to sort the data on a given column,
+and then take the top few in that ordering. You can use Miller’s
+then feature to pipe commands together.
Here are the records with the top three index values:
+$ mlr --icsv --opprint sort -f shape -nr index then head -n 3 example.csv +color shape flag index quantity rate +yellow circle 1 87 63.5058 8.3350 +yellow circle 1 73 63.9785 4.2370 +red circle 1 16 13.8103 2.9010 ++
Lots of Miller commands take a -g option
+for group-by: here, head -n 1 -g shape outputs the first record
+for each distinct value of the shape field. This means we’re
+finding the record with highest index field for each distinct
+shape field:
Statistics can be computed with or without group-by field(s). Also, the first of these two
-examples uses --oxtab output format which is a nice alternative to --opprint when you
-have lots of columns:
Statistics can be computed with or without group-by field(s)
- --$ mlr --icsv --oxtab --from example.csv stats1 -a p0,p10,p25,p50,p75,p90,p99,p100 -f rate -rate_p0 0.013000 -rate_p10 2.901000 -rate_p25 4.237000 -rate_p50 8.243000 -rate_p75 8.591000 -rate_p90 9.887000 -rate_p99 9.887000 -rate_p100 9.887000 --
@@ -456,6 +472,25 @@ square purple 1 72.373500 72.373500 72.373500
If your output has a lot of columns, you can use XTAB +format to line things up vertically for you instead:
+ + ++$ mlr --icsv --oxtab --from example.csv stats1 -a p0,p10,p25,p50,p75,p90,p99,p100 -f rate +rate_p0 0.013000 +rate_p10 2.901000 +rate_p25 4.237000 +rate_p50 8.243000 +rate_p75 8.591000 +rate_p90 9.887000 +rate_p99 9.887000 +rate_p100 9.887000 ++
Sample CSV data file:
+Suppose you have this CSV data file:
POKI_RUN_COMMAND{{cat example.csv}}HERE -mlr cat is like cat ...
mlr cat is like cat -- it passes the data
+through unmodified ...
... but it can also do format conversion (here, to pretty-printed tabular format):
+... but it can also do format conversion (here, you can pretty-print in tabular format):
POKI_RUN_COMMAND{{mlr --icsv --opprint cat example.csv}}HERE - mlr head and mlr tail count records rather than lines. The CSV
-header is included either way:
mlr head and mlr tail count
+records rather than lines. Whethere you’re getting the first few records
+or the last few, the CSV header is included either way:
Sort primarily alphabetically on one field, then secondarily -numerically descending on another field:
+You can sort primarily alphabetically on one field, then +secondarily numerically descending on another field:
POKI_RUN_COMMAND{{mlr --icsv --opprint sort -f shape -nr index example.csv}}HERE - Use cut to retain only specified fields, in input-data order:
You can use cut to retain only specified fields, in the
+same order they appeared in the input data:
Use cut -o to retain only specified fields, in your specified order:
You can also use cut -o to retain only
+specified fields in your preferred order:
Use cut -x to omit specified fields:
You can use cut -x to omit fields you
+don’t care about:
Use filter to retain specified records:
You can use filter to keep only records you care about:
Use put to add/replace fields which are computed from other fields:
You can use put to create new fields which
+are computed from other fields:
$[[3]] to access the name of field 3 or
POKI_RUN_COMMAND{{mlr --icsv --opprint put '$[[3]] = "NEW"' example.csv}}HERE
POKI_RUN_COMMAND{{mlr --icsv --opprint put '$[[[3]]] = "NEW"' example.csv}}HERE
-JSON output:
+OK, CSV and pretty-print are fine. But Miller can also convert between +a few other formats -- let’s take a look at JSON output:
POKI_RUN_COMMAND{{mlr --icsv --ojson put '$ratio = $quantity/$rate; $shape = toupper($shape)' example.csv}}HERE -JSON output with vertical-formatting flags:
+Or, JSON output with vertical-formatting flags:
-POKI_RUN_COMMAND{{mlr --icsv --ojson --jvstack --jlistwrap tail -n 2 example.csv}}HERE +POKI_RUN_COMMAND{{mlr --icsv --ojsonx tail -n 2 example.csv}}HERE - Use then to pipe commands together. Also, the
--g option for many Miller commands is for group-by: here, head -n
-1 -g shape outputs the first record for each distinct value of the
-shape field. This means we’re finding the record with highest
-index field for each distinct shape field:
Now suppose you want to sort the data on a given column,
+and then take the top few in that ordering. You can use Miller’s
+then feature to pipe commands together.
Here are the records with the top three index values:
Lots of Miller commands take a -g option
+for group-by: here, head -n 1 -g shape outputs the first record
+for each distinct value of the shape field. This means we’re
+finding the record with highest index field for each distinct
+shape field:
Statistics can be computed with or without group-by field(s). Also, the first of these two
-examples uses --oxtab output format which is a nice alternative to --opprint when you
-have lots of columns:
Statistics can be computed with or without group-by field(s)
-POKI_RUN_COMMAND{{mlr --icsv --oxtab --from example.csv stats1 -a p0,p10,p25,p50,p75,p90,p99,p100 -f rate}}HERE POKI_RUN_COMMAND{{mlr --icsv --opprint --from example.csv stats1 -a count,min,mean,max -f quantity -g shape}}HERE POKI_RUN_COMMAND{{mlr --icsv --opprint --from example.csv stats1 -a count,min,mean,max -f quantity -g shape,color}}HERE +If your output has a lot of columns, you can use XTAB +format to line things up vertically for you instead:
+ +POKI_RUN_COMMAND{{mlr --icsv --oxtab --from example.csv stats1 -a p0,p10,p25,p50,p75,p90,p99,p100 -f rate}}HERE +