Mirrors/miller
1
0
Fork 0
mirror of https://github.com/johnkerl/miller.git synced 2026-01-23 10:15:36 +00:00
Code Issues Projects Releases Wiki Activity

Datetime/timezones docs page (#707)

Browse source 
* xvright

* Left/right trim on --repifs

* Fix emit-flatten when main & emit output file formats differ

* Port localtime DSL functions from C to Go

* Port localtime DSL functions from C to Go (#703)

* xvright

* Left/right trim on --repifs

* Fix emit-flatten when main & emit output file formats differ

* Port localtime DSL functions from C to Go

* Time-related issues: 331, 332, 359, 477
This commit is contained in:
John Kerl 2021-10-20 23:48:43 -04:00 • committed by GitHub
parent 08fdd554f5
commit 2bca2d082b
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
7 changed files with 540 additions and 37 deletions
Download patch file Download diff file Expand all files Collapse all files

1
docs6/mkdocs.yml
View file

@ -93,6 +93,7 @@ nav:
- "DSL filter statements": "reference-dsl-filter-statements.md"
- "DSL output statements": "reference-dsl-output-statements.md"
- "DSL unset statements": "reference-dsl-unset-statements.md"
- "DSL datetime/timezone functions": "reference-dsl-time.md"
- "DSL errors and transparency": "reference-dsl-errors.md"
- "Differences from other programming languages": "reference-dsl-differences.md"
- "A note on the complexity of Miller's expression language": "reference-dsl-complexity.md"

2
docs6/src/record-heterogeneity.md
View file

@ -127,8 +127,6 @@ If you `mlr csv cat` this, you'll get an error message:
<b>mlr --csv cat data/het/ragged.csv</b>
</pre>
<pre class="pre-non-highlight-in-pair">
a,b,c
1,2,3
mlr : mlr: CSV header/data length mismatch 3 != 2 at filename data/het/ragged.csv row 3.
</pre>

309
docs6/src/reference-dsl-time.md Normal file
View file

@ -0,0 +1,309 @@
<!--- PLEASE DO NOT EDIT DIRECTLY. EDIT THE .md.in FILE PLEASE. --->
<div>
<span class="quicklinks">
Quick links:
&nbsp;
<a class="quicklink" href="../reference-main-flag-list/index.html">Flag list</a>
&nbsp;
<a class="quicklink" href="../reference-verbs/index.html">Verb list</a>
&nbsp;
<a class="quicklink" href="../reference-dsl-builtin-functions/index.html">Function list</a>
&nbsp;
<a class="quicklink" href="../glossary/index.html">Glossary</a>
&nbsp;
<a class="quicklink" href="https://github.com/johnkerl/miller" target="_blank">Repository ↗</a>
</span>
</div>
# DSL datetime/timezone functions
Dates/times are not a separate data type; Miller uses ints for
[seconds since the epoch](https://en.wikipedia.org/wiki/Unix_time) and strings for formatted
date/times. In this page we take a look at what some of the various options are
for processing datetimes andd timezones in your data.
See also the [section on time-related
functions](reference-dsl-builtin-functions/index.html#time-functions) for
information auto-generated from Miller's online-help strings.
# Epoch seconds
[Seconds since the epoch](https://en.wikipedia.org/wiki/Unix_time), or _Unix
Time_, is seconds (positive, zero, or negative) since midnight January 1 1970
UTC. This representation has several advantages, and is quite common in the
computing world.
Since this is a [number](reference-main-arithmetic.md) in Miller -- 64-bit
signed integer or double-precision floating-point -- this can represent dates
billions of years into the past or future without worry of overflow. (There is
no [year-2038 problem](https://en.wikipedia.org/wiki/Year_2038_problem) here.)
Being numbers, epoch-seconds are easy to store in databases, communicate over
networks in binary format, etc. Another benefit of epoch-seconds is that
they're independent of timezone or daylight-savings time.
One minus is that, being just numbers, they're not particularly human-readable
-- hence the to-string and from-string functions described below. Another
caveat (not really a minus) is that _epoch milliseconds_ are common in some
contexts, particulary JavaScript. If you every see a timestamp for the year
49,000-something -- probablhy someone is treating epoch-milliseconds as
epoch-seconds.
<pre class="pre-highlight-in-pair">
<b>mlr -n put 'end {</b>
<b> print sec2gmt(1500000000);</b>
<b> print sec2gmt(1500000000000);</b>
<b>}'</b>
</pre>
<pre class="pre-non-highlight-in-pair">
2017-07-14T02:40:00Z
49503-02-10T02:40:00Z
</pre>
You can get the current system time, as epoch-seconds, using the
[systime](reference-dsl-builtin-functions.md#systime) DSL function:
<pre class="pre-highlight-in-pair">
<b>mlr --c2p --from example.csv put '$t = systime()'</b>
</pre>
<pre class="pre-non-highlight-in-pair">
color shape flag k index quantity rate t
yellow triangle true 1 11 43.6498 9.8870 1634784588.045347
red square true 2 15 79.2778 0.0130 1634784588.045385
red circle true 3 16 13.8103 2.9010 1634784588.045386
red square false 4 48 77.5542 7.4670 1634784588.045393
purple triangle false 5 51 81.2290 8.5910 1634784588.045394
red square false 6 64 77.1991 9.5310 1634784588.045417
purple triangle false 7 65 80.1405 5.8240 1634784588.045418
yellow circle true 8 73 63.9785 4.2370 1634784588.045419
yellow circle true 9 87 63.5058 8.3350 1634784588.045421
purple square false 10 91 72.3735 8.2430 1634784588.045422
</pre>
The [systimeint](reference-dsl-builtin-functions.md#systimeint) DSL functions
is nothing more than a keystroke-saver for `int(systime())`.
# UTC times with standard format
One way to make epoch-seconds human-readable, while maintaining some of their
benefits such as being independent of timezone and daylight savings, is to use
the [ISO8601](https://en.wikipedia.org/wiki/ISO_8601) format. This was the
first (and initially only) human-readable date/time format supported by Miller
going all the way back to Miller 1.0.0.
You can get these from epoch-seconds using the
[sec2gmt](reference-dsl-builtin-functions.md#sec2gmt) DSL function.
(Note that the terms _UTC_ and _GMT_ are used interchangeably in Miller.)
We also have [sec2gmtdate](reference-dsl-builtin-functions.md#sec2gmtdate) DSL function.
<pre class="pre-highlight-in-pair">
<b>mlr -n put 'end {</b>
<b> print sec2gmt(0);</b>
<b> print sec2gmt(1234567890.123);</b>
<b> print sec2gmt(-1234567890.123);</b>
<b> print;</b>
<b> print sec2gmtdate(0);</b>
<b> print sec2gmtdate(1234567890.123);</b>
<b> print sec2gmtdate(-1234567890.123);</b>
<b>}'</b>
</pre>
<pre class="pre-non-highlight-in-pair">
1970-01-01T00:00:00Z
2009-02-13T23:31:30Z
1930-11-18T00:28:29Z
1970-01-01
2009-02-13
1930-11-18
</pre>
# Local times with standard format; specifying timezones
You can use similar formatting for dates in your preferred timezone, not just UTC/GMT.
We have the
[sec2localtime](reference-dsl-builtin-functions.md#sec2localtime),
[sec2localdate](reference-dsl-builtin-functions.md#sec2localdate), and
[localtime2sec](reference-dsl-builtin-functions.md#localtime2sec) DSL functions.
You can specify the timezone using any of the following:
* An environment variable, e.g. `export TZ=Asia/Istanbul` at your system prompt (`set TZ=Asia/Istanbul` in Windows)
* Using the `--tz` flag. This sets the `TZ` environment variable, but only internally to the `mlr` process
* Within a DSL expression, you can assign to `ENV["TZ"]`.
Regardless, if you specifiy an invalid timezone, you'll be clearly notified:
<pre class="pre-highlight-in-pair">
<b>mlr --from example.csv --tz This/Is/A/Typo cat</b>
</pre>
<pre class="pre-non-highlight-in-pair">
mlr: unknown time zone This/Is/A/Typo
</pre>
<pre class="pre-highlight-in-pair">
<b>export TZ=Asia/Istanbul</b>
<b>mlr -n put 'end { print sec2localtime(0) }'</b>
</pre>
<pre class="pre-non-highlight-in-pair">
1970-01-01 02:00:00
</pre>
<pre class="pre-highlight-in-pair">
<b>mlr --tz America/Sao_Paulo -n put 'end { print sec2localtime(0) }'</b>
</pre>
<pre class="pre-non-highlight-in-pair">
1969-12-31 21:00:00
</pre>
<pre class="pre-highlight-in-pair">
<b>mlr -n put 'end {</b>
<b> ENV["TZ"] = "Asia/Istanbul";</b>
<b> print sec2localtime(0);</b>
<b> print sec2localdate(0);</b>
<b> print localtime2sec("2000-01-02 03:04:05");</b>
<b> print;</b>
<b> ENV["TZ"] = "America/Sao_Paulo";</b>
<b> print sec2localtime(0);</b>
<b> print sec2localdate(0);</b>
<b> print localtime2sec("2000-01-02 03:04:05");</b>
<b>}'</b>
</pre>
<pre class="pre-non-highlight-in-pair">
1970-01-01 02:00:00
1970-01-01
946775045
1969-12-31 21:00:00
1969-12-31
946789445
</pre>
Note that for local times, Miller omits the `T` and the `Z` you see in GMT times.
We also have the
[gmt2localtime](reference-dsl-builtin-functions.md#gmt2localtime) and
[localtime2gmt](reference-dsl-builtin-functions.md#localtime2gmt) convenience functions:
<pre class="pre-highlight-in-pair">
<b>mlr -n put 'end {</b>
<b> ENV["TZ"] = "Asia/Istanbul";</b>
<b> print gmt2localtime("1970-01-01T00:00:00Z");</b>
<b> print localtime2gmt("1970-01-01 00:00:00");</b>
<b>}'</b>
</pre>
<pre class="pre-non-highlight-in-pair">
1970-01-01 02:00:00
1969-12-31T22:00:00Z
</pre>
# GMT and local times with custom formats
The to-string and from-string functions we've seen so far are low-keystroking:
with a little bit of typing you can convert datetimes to/from epoch seconds.
The minus, however, is flexibility. This is where the
[strftime](reference-dsl-builtin-functions.md#strftime),
[strptime](reference-dsl-builtin-functions.md#strptime) functions come into play.
Notes:
* The names `strftime` and `strptime` far predate Miller; they were chosen for familiarity. The `f` is for _format_: from epoch-seconds to human-readable string. The `p` is for _parse_: for doing the reverse.
* Even though Miller is written in Go as of Miller 6, it still preserves [C-like](https://en.wikipedia.org/wiki/C_date_and_time_functions#strftime) `strftime` and `strptime` semantics.
* For `strftime`, this is thanks to [https://github.com/lestrrat-go/strftime](https://github.com/lestrrat-go/strftime).
* For `stpftime`, this is thanks to [https://github.com/pbnjay/strptime](https://github.com/pbnjay/strptime).
* See [https://devhints.io/strftime](https://devhints.io/strftime) for sample format strings you can use.
Some examples:
<pre class="pre-highlight-in-pair">
<b>mlr -n put 'end {</b>
<b> ENV["TZ"] = "Asia/Istanbul";</b>
<b> print strftime(0, "%Y-%m-%d %H:%M:%S");</b>
<b> print strftime(0, "%Y-%m-%d %H:%M:%S %Z");</b>
<b> print strftime(0, "%Y-%m-%d %H:%M:%S %z");</b>
<b> print strftime(0, "%A, %B %e, %Y");</b>
<b> print strftime(123456789, "%I:%M %p");</b>
<b>}'</b>
</pre>
<pre class="pre-non-highlight-in-pair">
1970-01-01 00:00:00
1970-01-01 00:00:00 UTC
1970-01-01 00:00:00 +0000
Thursday, January 1, 1970
09:33 PM
</pre>
Unfortunately, names from `%A` and `%B` are only available in English, as an
artifact of a design choice in the Go `time` library which Miller (and its
`strftime` / `strptime` supporting packages as noted above) rely on.
We also have
[strftimelocal](reference-dsl-builtin-functions.md#strftimelocal) and
[strptimelocal](reference-dsl-builtin-functions.md#strptimelocal):
<pre class="pre-highlight-in-pair">
<b>mlr -n put 'end {</b>
<b> ENV["TZ"] = "America/Anchorage";</b>
<b> print strftime_local(0, "%Y-%m-%d %H:%M:%S %Z");</b>
<b> print strftime_local(0, "%Y-%m-%d %H:%M:%S %z");</b>
<b> print strftime_local(0, "%A, %B %e, %Y");</b>
<b> print strptime_local("2020-03-01 00:00:00", "%Y-%m-%d %H:%M:%S");</b>
<b> print;</b>
<b> ENV["TZ"] = "Asia/Hong_Kong";</b>
<b> print strftime_local(0, "%Y-%m-%d %H:%M:%S %Z");</b>
<b> print strftime_local(0, "%Y-%m-%d %H:%M:%S %z");</b>
<b> print strftime_local(0, "%A, %B %e, %Y");</b>
<b> print strptime_local("2020-03-01 00:00:00", "%Y-%m-%d %H:%M:%S");</b>
<b>}'</b>
</pre>
<pre class="pre-non-highlight-in-pair">
1969-12-31 14:00:00 AHST
1969-12-31 14:00:00 -1000
Wednesday, December 31, 1969
1583053200
1970-01-01 08:00:00 HKT
1970-01-01 08:00:00 +0800
Thursday, January 1, 1970
1582992000
</pre>
# Relative times
You can get the seconds since the Miller process start using
[uptime](reference-dsl-builtin-functions.md#uptime):
<pre class="pre-highlight-in-pair">
<b>color shape flag k index quantity rate u</b>
</pre>
<pre class="pre-non-highlight-in-pair">
yellow triangle true 1 11 43.6498 9.8870 0.0011110305786132812
red square true 2 15 79.2778 0.0130 0.0011241436004638672
red circle true 3 16 13.8103 2.9010 0.0011250972747802734
red square false 4 48 77.5542 7.4670 0.0011301040649414062
purple triangle false 5 51 81.2290 8.5910 0.0011301040649414062
red square false 6 64 77.1991 9.5310 0.002481222152709961
purple triangle false 7 65 80.1405 5.8240 0.0024831295013427734
yellow circle true 8 73 63.9785 4.2370 0.0024831295013427734
yellow circle true 9 87 63.5058 8.3350 0.0024852752685546875
purple square false 10 91 72.3735 8.2430 0.002485990524291992
</pre>
Time-differences can be done in seconds, of course; you can also use the following if you like:
<pre class="pre-highlight-in-pair">
<b>mlr -F | grep hms</b>
</pre>
<pre class="pre-non-highlight-in-pair">
dhms2fsec (class=time #args=1) Recovers floating-point seconds as in dhms2fsec("5d18h53m20.250000s") = 500000.250000
dhms2sec (class=time #args=1) Recovers integer seconds as in dhms2sec("5d18h53m20s") = 500000
fsec2dhms (class=time #args=1) Formats floating-point seconds as in fsec2dhms(500000.25) = "5d18h53m20.250000s"
fsec2hms (class=time #args=1) Formats floating-point seconds as in fsec2hms(5000.25) = "01:23:20.250000"
hms2fsec (class=time #args=1) Recovers floating-point seconds as in hms2fsec("01:23:20.250000") = 5000.250000
hms2sec (class=time #args=1) Recovers integer seconds as in hms2sec("01:23:20") = 5000
sec2dhms (class=time #args=1) Formats integer seconds as in sec2dhms(500000) = "5d18h53m20s"
sec2hms (class=time #args=1) Formats integer seconds as in sec2hms(5000) = "01:23:20"
</pre>
# References
* List of formatting characters for `strftime` and `strptime`: [https://devhints.io/strftime](https://devhints.io/strftime)
* List of valid timezone names: [https://en.wikipedia.org/wiki/List_of_tz_database_time_zones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)

226
docs6/src/reference-dsl-time.md.in Normal file
View file

@ -0,0 +1,226 @@
# DSL datetime/timezone functions
Dates/times are not a separate data type; Miller uses ints for
[seconds since the epoch](https://en.wikipedia.org/wiki/Unix_time) and strings for formatted
date/times. In this page we take a look at what some of the various options are
for processing datetimes andd timezones in your data.
See also the [section on time-related
functions](reference-dsl-builtin-functions/index.html#time-functions) for
information auto-generated from Miller's online-help strings.
# Epoch seconds
[Seconds since the epoch](https://en.wikipedia.org/wiki/Unix_time), or _Unix
Time_, is seconds (positive, zero, or negative) since midnight January 1 1970
UTC. This representation has several advantages, and is quite common in the
computing world.
Since this is a [number](reference-main-arithmetic.md) in Miller -- 64-bit
signed integer or double-precision floating-point -- this can represent dates
billions of years into the past or future without worry of overflow. (There is
no [year-2038 problem](https://en.wikipedia.org/wiki/Year_2038_problem) here.)
Being numbers, epoch-seconds are easy to store in databases, communicate over
networks in binary format, etc. Another benefit of epoch-seconds is that
they're independent of timezone or daylight-savings time.
One minus is that, being just numbers, they're not particularly human-readable
-- hence the to-string and from-string functions described below. Another
caveat (not really a minus) is that _epoch milliseconds_ are common in some
contexts, particulary JavaScript. If you every see a timestamp for the year
49,000-something -- probablhy someone is treating epoch-milliseconds as
epoch-seconds.
GENMD_RUN_COMMAND
mlr -n put 'end {
print sec2gmt(1500000000);
print sec2gmt(1500000000000);
}'
GENMD_EOF
You can get the current system time, as epoch-seconds, using the
[systime](reference-dsl-builtin-functions.md#systime) DSL function:
GENMD_CARDIFY_HIGHLIGHT_ONE
mlr --c2p --from example.csv put '$t = systime()'
color shape flag k index quantity rate t
yellow triangle true 1 11 43.6498 9.8870 1634784588.045347
red square true 2 15 79.2778 0.0130 1634784588.045385
red circle true 3 16 13.8103 2.9010 1634784588.045386
red square false 4 48 77.5542 7.4670 1634784588.045393
purple triangle false 5 51 81.2290 8.5910 1634784588.045394
red square false 6 64 77.1991 9.5310 1634784588.045417
purple triangle false 7 65 80.1405 5.8240 1634784588.045418
yellow circle true 8 73 63.9785 4.2370 1634784588.045419
yellow circle true 9 87 63.5058 8.3350 1634784588.045421
purple square false 10 91 72.3735 8.2430 1634784588.045422
GENMD_EOF
The [systimeint](reference-dsl-builtin-functions.md#systimeint) DSL functions
is nothing more than a keystroke-saver for `int(systime())`.
# UTC times with standard format
One way to make epoch-seconds human-readable, while maintaining some of their
benefits such as being independent of timezone and daylight savings, is to use
the [ISO8601](https://en.wikipedia.org/wiki/ISO_8601) format. This was the
first (and initially only) human-readable date/time format supported by Miller
going all the way back to Miller 1.0.0.
You can get these from epoch-seconds using the
[sec2gmt](reference-dsl-builtin-functions.md#sec2gmt) DSL function.
(Note that the terms _UTC_ and _GMT_ are used interchangeably in Miller.)
We also have [sec2gmtdate](reference-dsl-builtin-functions.md#sec2gmtdate) DSL function.
GENMD_RUN_COMMAND
mlr -n put 'end {
print sec2gmt(0);
print sec2gmt(1234567890.123);
print sec2gmt(-1234567890.123);
print;
print sec2gmtdate(0);
print sec2gmtdate(1234567890.123);
print sec2gmtdate(-1234567890.123);
}'
GENMD_EOF
# Local times with standard format; specifying timezones
You can use similar formatting for dates in your preferred timezone, not just UTC/GMT.
We have the
[sec2localtime](reference-dsl-builtin-functions.md#sec2localtime),
[sec2localdate](reference-dsl-builtin-functions.md#sec2localdate), and
[localtime2sec](reference-dsl-builtin-functions.md#localtime2sec) DSL functions.
You can specify the timezone using any of the following:
* An environment variable, e.g. `export TZ=Asia/Istanbul` at your system prompt (`set TZ=Asia/Istanbul` in Windows)
* Using the `--tz` flag. This sets the `TZ` environment variable, but only internally to the `mlr` process
* Within a DSL expression, you can assign to `ENV["TZ"]`.
Regardless, if you specifiy an invalid timezone, you'll be clearly notified:
GENMD_RUN_COMMAND_TOLERATING_ERROR
mlr --from example.csv --tz This/Is/A/Typo cat
GENMD_EOF
GENMD_RUN_COMMAND
export TZ=Asia/Istanbul
mlr -n put 'end { print sec2localtime(0) }'
GENMD_EOF
GENMD_RUN_COMMAND
mlr --tz America/Sao_Paulo -n put 'end { print sec2localtime(0) }'
GENMD_EOF
GENMD_RUN_COMMAND
mlr -n put 'end {
ENV["TZ"] = "Asia/Istanbul";
print sec2localtime(0);
print sec2localdate(0);
print localtime2sec("2000-01-02 03:04:05");
print;
ENV["TZ"] = "America/Sao_Paulo";
print sec2localtime(0);
print sec2localdate(0);
print localtime2sec("2000-01-02 03:04:05");
}'
GENMD_EOF
Note that for local times, Miller omits the `T` and the `Z` you see in GMT times.
We also have the
[gmt2localtime](reference-dsl-builtin-functions.md#gmt2localtime) and
[localtime2gmt](reference-dsl-builtin-functions.md#localtime2gmt) convenience functions:
GENMD_RUN_COMMAND
mlr -n put 'end {
ENV["TZ"] = "Asia/Istanbul";
print gmt2localtime("1970-01-01T00:00:00Z");
print localtime2gmt("1970-01-01 00:00:00");
}'
GENMD_EOF
# GMT and local times with custom formats
The to-string and from-string functions we've seen so far are low-keystroking:
with a little bit of typing you can convert datetimes to/from epoch seconds.
The minus, however, is flexibility. This is where the
[strftime](reference-dsl-builtin-functions.md#strftime),
[strptime](reference-dsl-builtin-functions.md#strptime) functions come into play.
Notes:
* The names `strftime` and `strptime` far predate Miller; they were chosen for familiarity. The `f` is for _format_: from epoch-seconds to human-readable string. The `p` is for _parse_: for doing the reverse.
* Even though Miller is written in Go as of Miller 6, it still preserves [C-like](https://en.wikipedia.org/wiki/C_date_and_time_functions#strftime) `strftime` and `strptime` semantics.
* For `strftime`, this is thanks to [https://github.com/lestrrat-go/strftime](https://github.com/lestrrat-go/strftime).
* For `stpftime`, this is thanks to [https://github.com/pbnjay/strptime](https://github.com/pbnjay/strptime).
* See [https://devhints.io/strftime](https://devhints.io/strftime) for sample format strings you can use.
Some examples:
GENMD_RUN_COMMAND
mlr -n put 'end {
ENV["TZ"] = "Asia/Istanbul";
print strftime(0, "%Y-%m-%d %H:%M:%S");
print strftime(0, "%Y-%m-%d %H:%M:%S %Z");
print strftime(0, "%Y-%m-%d %H:%M:%S %z");
print strftime(0, "%A, %B %e, %Y");
print strftime(123456789, "%I:%M %p");
}'
GENMD_EOF
Unfortunately, names from `%A` and `%B` are only available in English, as an
artifact of a design choice in the Go `time` library which Miller (and its
`strftime` / `strptime` supporting packages as noted above) rely on.
We also have
[strftimelocal](reference-dsl-builtin-functions.md#strftimelocal) and
[strptimelocal](reference-dsl-builtin-functions.md#strptimelocal):
GENMD_RUN_COMMAND
mlr -n put 'end {
ENV["TZ"] = "America/Anchorage";
print strftime_local(0, "%Y-%m-%d %H:%M:%S %Z");
print strftime_local(0, "%Y-%m-%d %H:%M:%S %z");
print strftime_local(0, "%A, %B %e, %Y");
print strptime_local("2020-03-01 00:00:00", "%Y-%m-%d %H:%M:%S");
print;
ENV["TZ"] = "Asia/Hong_Kong";
print strftime_local(0, "%Y-%m-%d %H:%M:%S %Z");
print strftime_local(0, "%Y-%m-%d %H:%M:%S %z");
print strftime_local(0, "%A, %B %e, %Y");
print strptime_local("2020-03-01 00:00:00", "%Y-%m-%d %H:%M:%S");
}'
GENMD_EOF
# Relative times
You can get the seconds since the Miller process start using
[uptime](reference-dsl-builtin-functions.md#uptime):
GENMD_CARDIFY_HIGHLIGHT_ONE
color shape flag k index quantity rate u
yellow triangle true 1 11 43.6498 9.8870 0.0011110305786132812
red square true 2 15 79.2778 0.0130 0.0011241436004638672
red circle true 3 16 13.8103 2.9010 0.0011250972747802734
red square false 4 48 77.5542 7.4670 0.0011301040649414062
purple triangle false 5 51 81.2290 8.5910 0.0011301040649414062
red square false 6 64 77.1991 9.5310 0.002481222152709961
purple triangle false 7 65 80.1405 5.8240 0.0024831295013427734
yellow circle true 8 73 63.9785 4.2370 0.0024831295013427734
yellow circle true 9 87 63.5058 8.3350 0.0024852752685546875
purple square false 10 91 72.3735 8.2430 0.002485990524291992
GENMD_EOF
Time-differences can be done in seconds, of course; you can also use the following if you like:
GENMD_RUN_COMMAND
mlr -F | grep hms
GENMD_EOF
# References
* List of formatting characters for `strftime` and `strptime`: [https://devhints.io/strftime](https://devhints.io/strftime)
* List of valid timezone names: [https://en.wikipedia.org/wiki/List_of_tz_database_time_zones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)

1
docs6/src/reference-main-data-types.md
View file

@ -23,6 +23,7 @@ Miller's types are:
* Scalars:
* **string**: such as `"abcdefg"`, supporting concatenation, one-up indexing and slicing, and [library functions](reference-dsl-builtin-functions.md#string-functions). See the pages on [strings](reference-main-strings.md) and [regular expressions](reference-main-regular-expressions.md).
* **float** and **int**: such as `1.2` and `3`: double-precision and 64-bit signed, respectively. See the section on [arithmetic operators and math-related library functions](reference-dsl-builtin-functions.md#math-functions) as well as the [Arithmetic](reference-main-arithmetic.md) page.
* dates/times are not a separate data type; Miller uses ints for [seconds since the epoch](https://en.wikipedia.org/wiki/Unix_time) and strings for formatted date/times. See the [DSL datetime/timezone functions page](reference-dsl-time.md) for more information.
* **boolean**: literals `true` and `false`; results of `==`, `<`, `>`, etc. See the section on [boolean operators](reference-dsl-builtin-functions.md#boolean-functions).
* Collections:
* **map**: such as `{"a":1,"b":[2,3,4]}`, supporting key-indexing, preservation of insertion order, [library functions](reference-dsl-builtin-functions.md#collections-functions), etc. See the [Maps](reference-main-maps.md) page.

1
docs6/src/reference-main-data-types.md.in
View file

@ -7,6 +7,7 @@ Miller's types are:
* Scalars:
* **string**: such as `"abcdefg"`, supporting concatenation, one-up indexing and slicing, and [library functions](reference-dsl-builtin-functions.md#string-functions). See the pages on [strings](reference-main-strings.md) and [regular expressions](reference-main-regular-expressions.md).
* **float** and **int**: such as `1.2` and `3`: double-precision and 64-bit signed, respectively. See the section on [arithmetic operators and math-related library functions](reference-dsl-builtin-functions.md#math-functions) as well as the [Arithmetic](reference-main-arithmetic.md) page.
* dates/times are not a separate data type; Miller uses ints for [seconds since the epoch](https://en.wikipedia.org/wiki/Unix_time) and strings for formatted date/times. See the [DSL datetime/timezone functions page](reference-dsl-time.md) for more information.
* **boolean**: literals `true` and `false`; results of `==`, `<`, `>`, etc. See the section on [boolean operators](reference-dsl-builtin-functions.md#boolean-functions).
* Collections:
* **map**: such as `{"a":1,"b":[2,3,4]}`, supporting key-indexing, preservation of insertion order, [library functions](reference-dsl-builtin-functions.md#collections-functions), etc. See the [Maps](reference-main-maps.md) page.

37
go/todo.txt
View file

@ -56,6 +56,8 @@ d pre-release:
o draft release w/ m6 binaries YYYYMMDD -- ?
o gha download steps clearer
* wut h1 spacing before/after ...
----------------------------------------------------------------
LOCALTIME
@ -74,44 +76,9 @@ LOCALTIME
* new webdoc separate PR
- incl https://github.com/johnkerl/miller/issues/331
* ENV["TZ"] update:
o --tz is a setenv & is treated the same. also ENV["TZ"]
o no explicit tz arg @ dsl functions, sorry ...
o now maybe simplify the RTs ...
o at very least, at a multi-tz case
o sec2gmt sec2localtime
sec2gmtdate sec2localdate
strftime strftime_local
o gmt2sec localtime2sec
strptime strptime_local
o gmt2localtime localtime2gmt
o note in new webdoc
mlr> sec2localtime(0)
"1969-12-31 19:00:00"
mlr> ENV["TZ"]=""
mlr> sec2localtime(0)
"1970-01-01 00:00:00"
mlr> ENV["TZ"]="GMT"
mlr> sec2localtime(0)
"1970-01-01 00:00:00"
mlr> ENV["TZ"]="Asia/Istanbul"
mlr> sec2localtime(0)
"1970-01-01 02:00:00"
mlr> ENV["TZ"]="America/Sao_Paulo"
mlr> sec2localtime(0)
"1969-12-31 21:00:00"
o ref:
https://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html
http://www.iana.org/time-zones
https://en.wikipedia.org/wiki/Tz_database
https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
* https://github.com/johnkerl/miller/issues/332
o localtime2gmt and gmt2localtime
--tz
k maybe health-check with `location, err := time.LoadLocation("America/Los_Angeles")` -- ?
? make this this applies to all datetimes that don't include a time zone
? also make sure that, absent this, all zoneless datatimes are interpreted in that timezone
----------------------------------------------------------------
398