From 20dc9f0151eeb8bd0d06f6827e07baed76678fd3 Mon Sep 17 00:00:00 2001 From: John Kerl Date: Fri, 27 Oct 2017 10:28:00 -0700 Subject: [PATCH] data-sharing doc page: DKVP in Ruby and Python --- .gitignore | 1 + doc/10-min.html | 1 + doc/build.html | 1 + doc/contact.html | 1 + doc/content-for-data-sharing.html | 58 ++++ doc/cookbook.html | 1 + doc/cookbook2.html | 1 + doc/cookbook3.html | 1 + doc/data-examples.html | 1 + doc/data-sharing.html | 490 ++++++++++++++++++++++++++++++ doc/etymology.html | 1 + doc/faq.html | 1 + doc/feature-comparison.html | 1 + doc/file-formats.html | 1 + doc/index.html | 1 + doc/internationalization.html | 1 + doc/manpage.html | 1 + doc/originality.html | 1 + doc/performance.html | 1 + doc/poki.cfg | 1 + doc/polyglot-dkvp-io/dkvp_io.py | 58 ++++ doc/polyglot-dkvp-io/dkvp_io.rb | 52 ++++ doc/polyglot-dkvp-io/example.py | 31 ++ doc/polyglot-dkvp-io/example.rb | 24 ++ doc/record-heterogeneity.html | 1 + doc/reference-dsl.html | 1 + doc/reference-verbs.html | 1 + doc/reference.html | 1 + doc/release-docs.html | 1 + doc/to-do.html | 1 + doc/why.html | 1 + doc/whyc.html | 1 + 32 files changed, 739 insertions(+) create mode 100644 doc/content-for-data-sharing.html create mode 100644 doc/data-sharing.html create mode 100644 doc/polyglot-dkvp-io/dkvp_io.py create mode 100644 doc/polyglot-dkvp-io/dkvp_io.rb create mode 100644 doc/polyglot-dkvp-io/example.py create mode 100644 doc/polyglot-dkvp-io/example.rb diff --git a/.gitignore b/.gitignore index 7e0744274..23591aa5c 100644 --- a/.gitignore +++ b/.gitignore @@ -40,6 +40,7 @@ catc0 catm gmon.out *.o +*.pyc .swp .swo .*.swp diff --git a/doc/10-min.html b/doc/10-min.html index aa65f8bee..89cc952f7 100644 --- a/doc/10-min.html +++ b/doc/10-min.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/build.html b/doc/build.html index 8922f900f..e5d838310 100644 --- a/doc/build.html +++ b/doc/build.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/contact.html b/doc/contact.html index c980ca796..4074da6e5 100644 --- a/doc/contact.html +++ b/doc/contact.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/content-for-data-sharing.html b/doc/content-for-data-sharing.html new file mode 100644 index 000000000..1776839c0 --- /dev/null +++ b/doc/content-for-data-sharing.html @@ -0,0 +1,58 @@ +POKI_PUT_TOC_HERE + +

+ + + +

As discussed in the section on +POKI_PUT_LINK_FOR_PAGE(file-formats.html)HERE, Miller supports several +different file formats. Different tools are good at different things, so +it’s important to be able to move data into and out of other languages. +CSV and JSON are well-known, of course; here are some examples using DKVP +format, with Ruby and Python. + + +

DKVP I/O in Python

+ +
+ +

+Here are the I/O routines: + +POKI_INCLUDE_ESCAPED(polyglot-dkvp-io/dkvp_io.py)HERE + +And here is an example using them: + +POKI_RUN_COMMAND{{cat polyglot-dkvp-io/example.py}}HERE + +Run as-is: + +POKI_RUN_COMMAND{{python polyglot-dkvp-io/example.py < data/small}}HERE + +Run as-is, then pipe to Miller for pretty-printing: + +POKI_RUN_COMMAND{{python polyglot-dkvp-io/example.py < data/small | mlr --opprint cat}}HERE + +

+

DKVP I/O in Ruby

+ +
+ +

+Here are the I/O routines: + +POKI_INCLUDE_ESCAPED(polyglot-dkvp-io/dkvp_io.rb)HERE + +And here is an example using them: + +POKI_RUN_COMMAND{{cat polyglot-dkvp-io/example.rb}}HERE + +Run as-is: + +POKI_RUN_COMMAND{{ruby -I./polyglot-dkvp-io polyglot-dkvp-io/example.rb data/small}}HERE + +Run as-is, then pipe to Miller for pretty-printing: + +POKI_RUN_COMMAND{{ruby -I./polyglot-dkvp-io polyglot-dkvp-io/example.rb data/small | mlr --opprint cat}}HERE + +

diff --git a/doc/cookbook.html b/doc/cookbook.html index e95826705..a315044e4 100644 --- a/doc/cookbook.html +++ b/doc/cookbook.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/cookbook2.html b/doc/cookbook2.html index bc1c3863a..f43244190 100644 --- a/doc/cookbook2.html +++ b/doc/cookbook2.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/cookbook3.html b/doc/cookbook3.html index efc068423..98f4cd482 100644 --- a/doc/cookbook3.html +++ b/doc/cookbook3.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/data-examples.html b/doc/data-examples.html index 809aa0269..1631f8f37 100644 --- a/doc/data-examples.html +++ b/doc/data-examples.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/data-sharing.html b/doc/data-sharing.html new file mode 100644 index 000000000..d48fa5f5c --- /dev/null +++ b/doc/data-sharing.html @@ -0,0 +1,490 @@ + + + + + + + + + + + + Sharing data with other languages + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
Miller
+ + +
Overview: +
• About Miller +
• Miller in 10 minutes +
• File formats +
• Miller features in the context of the Unix toolkit +
• Record-heterogeneity +
• Internationalization +
Using Miller: +
• FAQ +
• Sharing data with other languages +
• Cookbook part 1 +
• Cookbook part 2 +
• Cookbook part 3 +
• Data-diving examples +
• Manpage +
• Reference +
• Reference: Verbs +
• Reference: DSL +
• Documents by release +
• Installation, portability, dependencies, and testing +
Background: +
• Why? +
• Why C? +
• Why call it Miller? +
• How original is Miller? +
• Performance +
Repository: +
• Things to do +
• Contact information +
• GitHub repo +











+











+





+
+ 		+ + + +
+
Sharing data with other languages
+

+ + +

+
Contents:
+• DKVP I/O in Python
+• DKVP I/O in Ruby
+
+

+ +

+ + + +

As discussed in the section on +File formats, Miller supports several +different file formats. Different tools are good at different things, so +it’s important to be able to move data into and out of other languages. +CSV and JSON are well-known, of course; here are some examples using DKVP +format, with Ruby and Python. + + +

DKVP I/O in Python

+ +
+ +

+Here are the I/O routines: + +

+

+
+#!/usr/bin/env python
+
+# ================================================================
+# Example of DKVP I/O using Python.
+#
+# Key point: Use Miller for what it's good at; pass data into/out of tools in
+# other languages to do what they're good at.
+#
+#   bash$ python -i dkvp_io.py
+#
+#   # READ
+#   >>> map = dkvpline2map('x=1,y=2', '=', ',')
+#   >>> map
+#   OrderedDict([('x', '1'), ('y', '2')])
+#
+#   # MODIFY
+#   >>> map['z'] = map['x'] + map['y']
+#   >>> map
+#   OrderedDict([('x', '1'), ('y', '2'), ('z', 3)])
+#
+#   # WRITE
+#   >>> line = map2dkvpline(map, '=', ',')
+#   >>> line
+#   'x=1,y=2,z=3'
+#
+# ================================================================
+
+import re
+import collections
+
+# ----------------------------------------------------------------
+# ips and ifs (input pair separator and input field separator) are nominally '=' and ','.
+def dkvpline2map(line, ips, ifs):
+	pairs = re.split(ifs, line)
+	map = collections.OrderedDict()
+	for pair in pairs:
+		key, value = re.split(ips, pair, 1)
+
+		# Type inference:
+		try:
+			value = int(value)
+		except:
+			try:
+				value = float(value)
+			except:
+				pass
+
+		map[key] = value
+	return map
+
+# ----------------------------------------------------------------
+# ops and ofs (output pair separator and output field separator) are nominally '=' and ','.
+def map2dkvpline(map , ops, ofs):
+	line = ''
+	pairs = []
+	for key in map:
+		pairs.append(str(key) + ops + str(map[key]))
+	return str.join(ofs, pairs)
+
+
+

+ +And here is an example using them: + +

+

+
+$ cat polyglot-dkvp-io/example.py
+#!/usr/bin/env ruby
+
+import sys
+import re
+import dkvp_io
+
+while True:
+	# Read the original record:
+	line = sys.stdin.readline().strip()
+	if line == '':
+		break
+	map = dkvp_io.dkvpline2map(line, '=', ',')
+
+	# Drop a field:
+	map.pop('x')
+
+	# Compute some new fields:
+	map['ab'] = map['a'] + map['b']
+	map['iy'] = map['i'] + map['y']
+
+	# Add new fields which show type of each already-existing field:
+	keys = map.keys()
+	for key in keys:
+		# Convert "<type 'int'>" to just "int", etc.:
+		type_string = str(map[key].__class__)
+		type_string = re.sub("<type '", "", type_string)
+		type_string = re.sub("'>", "", type_string)
+		map['t'+key] = type_string
+
+	# Write the modified record:
+	print dkvp_io.map2dkvpline(map, '=', ',')
+
+
+

+ +Run as-is: + +

+

+
+$ python polyglot-dkvp-io/example.py < data/small
+a=pan,b=pan,i=1,y=0.726802862743,ab=panpan,iy=1.72680286274,ta=str,tb=str,ti=int,ty=float,tab=str,tiy=float
+a=eks,b=pan,i=2,y=0.522151108333,ab=ekspan,iy=2.52215110833,ta=str,tb=str,ti=int,ty=float,tab=str,tiy=float
+a=wye,b=wye,i=3,y=0.338318525517,ab=wyewye,iy=3.33831852552,ta=str,tb=str,ti=int,ty=float,tab=str,tiy=float
+a=eks,b=wye,i=4,y=0.134188743284,ab=ekswye,iy=4.13418874328,ta=str,tb=str,ti=int,ty=float,tab=str,tiy=float
+a=wye,b=pan,i=5,y=0.863624469903,ab=wyepan,iy=5.8636244699,ta=str,tb=str,ti=int,ty=float,tab=str,tiy=float
+
+
+

+ +Run as-is, then pipe to Miller for pretty-printing: + +

+

+
+$ python polyglot-dkvp-io/example.py < data/small | mlr --opprint cat
+a   b   i y              ab     iy            ta  tb  ti  ty    tab tiy
+pan pan 1 0.726802862743 panpan 1.72680286274 str str int float str float
+eks pan 2 0.522151108333 ekspan 2.52215110833 str str int float str float
+wye wye 3 0.338318525517 wyewye 3.33831852552 str str int float str float
+eks wye 4 0.134188743284 ekswye 4.13418874328 str str int float str float
+wye pan 5 0.863624469903 wyepan 5.8636244699  str str int float str float
+
+
+

+ +

+

DKVP I/O in Ruby

+ +
+ +

+Here are the I/O routines: + +

+

+
+#!/usr/bin/env ruby
+
+# ================================================================
+# Example of DKVP I/O using Ruby.
+#
+# Key point: Use Miller for what it's good at; pass data into/out of tools in
+# other languages to do what they're good at.
+#
+#   bash$ irb -I. -r dkvp_io.rb
+#
+#   # READ
+#   irb(main):001:0> map = dkvpline2map('x=1,y=2', '=', ',')
+#   => {"x"=>"1", "y"=>"2"}
+#
+#   # MODIFY
+#   irb(main):001:0> map['z'] = map['x'] + map['y']
+#   => 3
+#
+#   # WRITE
+#   irb(main):002:0> line = map2dkvpline(map, '=', ',')
+#   => "x=1,y=2,z=3"
+#
+# ================================================================
+
+# ----------------------------------------------------------------
+# ips and ifs (input pair separator and input field separator) are nominally '=' and ','.
+def dkvpline2map(line, ips, ifs)
+  map = {}
+  line.split(ifs).each do |pair|
+    (k, v) = pair.split(ips, 2)
+
+    # Type inference:
+    begin
+      v = Integer(v)
+    rescue ArgumentError
+      begin
+        v = Float(v)
+      rescue ArgumentError
+        # Leave as string
+      end
+    end
+
+    map[k] = v
+  end
+  map
+end
+
+# ----------------------------------------------------------------
+# ops and ofs (output pair separator and output field separator) are nominally '=' and ','.
+def map2dkvpline(map, ops, ofs)
+  map.collect{|k,v| k.to_s + ops + v.to_s}.join(ofs)
+end
+
+
+

+ +And here is an example using them: + +

+

+
+$ cat polyglot-dkvp-io/example.rb
+#!/usr/bin/env ruby
+
+require 'dkvp_io'
+
+ARGF.each do |line|
+  # Read the original record:
+  map = dkvpline2map(line.chomp, '=', ',')
+
+  # Drop a field:
+  map.delete('x')
+
+  # Compute some new fields:
+  map['ab'] = map['a'] + map['b']
+  map['iy'] = map['i'] + map['y']
+
+  # Add new fields which show type of each already-existing field:
+  keys = map.keys
+  keys.each do |key|
+    map['t'+key] = map[key].class
+  end
+
+  # Write the modified record:
+  puts map2dkvpline(map, '=', ',')
+end
+
+
+

+ +Run as-is: + +

+

+
+$ ruby -I./polyglot-dkvp-io polyglot-dkvp-io/example.rb data/small
+a=pan,b=pan,i=1,y=0.7268028627434533,ab=panpan,iy=1.7268028627434533,ta=String,tb=String,ti=Fixnum,ty=Float,tab=String,tiy=Float
+a=eks,b=pan,i=2,y=0.5221511083334797,ab=ekspan,iy=2.5221511083334796,ta=String,tb=String,ti=Fixnum,ty=Float,tab=String,tiy=Float
+a=wye,b=wye,i=3,y=0.33831852551664776,ab=wyewye,iy=3.3383185255166477,ta=String,tb=String,ti=Fixnum,ty=Float,tab=String,tiy=Float
+a=eks,b=wye,i=4,y=0.13418874328430463,ab=ekswye,iy=4.134188743284304,ta=String,tb=String,ti=Fixnum,ty=Float,tab=String,tiy=Float
+a=wye,b=pan,i=5,y=0.8636244699032729,ab=wyepan,iy=5.863624469903273,ta=String,tb=String,ti=Fixnum,ty=Float,tab=String,tiy=Float
+
+
+

+ +Run as-is, then pipe to Miller for pretty-printing: + +

+

+
+$ ruby -I./polyglot-dkvp-io polyglot-dkvp-io/example.rb data/small | mlr --opprint cat
+a   b   i y                   ab     iy                 ta     tb     ti     ty    tab    tiy
+pan pan 1 0.7268028627434533  panpan 1.7268028627434533 String String Fixnum Float String Float
+eks pan 2 0.5221511083334797  ekspan 2.5221511083334796 String String Fixnum Float String Float
+wye wye 3 0.33831852551664776 wyewye 3.3383185255166477 String String Fixnum Float String Float
+eks wye 4 0.13418874328430463 ekswye 4.134188743284304  String String Fixnum Float String Float
+wye pan 5 0.8636244699032729  wyepan 5.863624469903273  String String Fixnum Float String Float
+
+
+

+ +

+
+
+ + diff --git a/doc/etymology.html b/doc/etymology.html index e3fd06d33..135bb01bd 100644 --- a/doc/etymology.html +++ b/doc/etymology.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/faq.html b/doc/faq.html index e1c81efcf..e7bac8373 100644 --- a/doc/faq.html +++ b/doc/faq.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/feature-comparison.html b/doc/feature-comparison.html index ad3f59a33..bdea41fe6 100644 --- a/doc/feature-comparison.html +++ b/doc/feature-comparison.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/file-formats.html b/doc/file-formats.html index 1973542cd..ad5440572 100644 --- a/doc/file-formats.html +++ b/doc/file-formats.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/index.html b/doc/index.html index d9e11790a..70341e987 100644 --- a/doc/index.html +++ b/doc/index.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/internationalization.html b/doc/internationalization.html index 8a8d693e2..61ccb65c9 100644 --- a/doc/internationalization.html +++ b/doc/internationalization.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/manpage.html b/doc/manpage.html index 7b5b2ef86..591592d11 100644 --- a/doc/manpage.html +++ b/doc/manpage.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/originality.html b/doc/originality.html index 235ed28ec..bc6474100 100644 --- a/doc/originality.html +++ b/doc/originality.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/performance.html b/doc/performance.html index c308e2dbc..e060aeee4 100644 --- a/doc/performance.html +++ b/doc/performance.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/poki.cfg b/doc/poki.cfg index b8e21d869..cb50e5595 100644 --- a/doc/poki.cfg +++ b/doc/poki.cfg @@ -8,6 +8,7 @@ internationalization.html Internationalization sep:details Using Miller: faq.html FAQ +data-sharing.html Sharing data with other languages cookbook.html Cookbook part 1 cookbook2.html Cookbook part 2 cookbook3.html Cookbook part 3 diff --git a/doc/polyglot-dkvp-io/dkvp_io.py b/doc/polyglot-dkvp-io/dkvp_io.py new file mode 100644 index 000000000..40828f51a --- /dev/null +++ b/doc/polyglot-dkvp-io/dkvp_io.py @@ -0,0 +1,58 @@ +#!/usr/bin/env python + +# ================================================================ +# Example of DKVP I/O using Python. +# +# Key point: Use Miller for what it's good at; pass data into/out of tools in +# other languages to do what they're good at. +# +# bash$ python -i dkvp_io.py +# +# # READ +# >>> map = dkvpline2map('x=1,y=2', '=', ',') +# >>> map +# OrderedDict([('x', '1'), ('y', '2')]) +# +# # MODIFY +# >>> map['z'] = map['x'] + map['y'] +# >>> map +# OrderedDict([('x', '1'), ('y', '2'), ('z', 3)]) +# +# # WRITE +# >>> line = map2dkvpline(map, '=', ',') +# >>> line +# 'x=1,y=2,z=3' +# +# ================================================================ + +import re +import collections + +# ---------------------------------------------------------------- +# ips and ifs (input pair separator and input field separator) are nominally '=' and ','. +def dkvpline2map(line, ips, ifs): + pairs = re.split(ifs, line) + map = collections.OrderedDict() + for pair in pairs: + key, value = re.split(ips, pair, 1) + + # Type inference: + try: + value = int(value) + except: + try: + value = float(value) + except: + pass + + map[key] = value + return map + +# ---------------------------------------------------------------- +# ops and ofs (output pair separator and output field separator) are nominally '=' and ','. +def map2dkvpline(map , ops, ofs): + line = '' + pairs = [] + for key in map: + pairs.append(str(key) + ops + str(map[key])) + return str.join(ofs, pairs) diff --git a/doc/polyglot-dkvp-io/dkvp_io.rb b/doc/polyglot-dkvp-io/dkvp_io.rb new file mode 100644 index 000000000..6a852f69a --- /dev/null +++ b/doc/polyglot-dkvp-io/dkvp_io.rb @@ -0,0 +1,52 @@ +#!/usr/bin/env ruby + +# ================================================================ +# Example of DKVP I/O using Ruby. +# +# Key point: Use Miller for what it's good at; pass data into/out of tools in +# other languages to do what they're good at. +# +# bash$ irb -I. -r dkvp_io.rb +# +# # READ +# irb(main):001:0> map = dkvpline2map('x=1,y=2', '=', ',') +# => {"x"=>"1", "y"=>"2"} +# +# # MODIFY +# irb(main):001:0> map['z'] = map['x'] + map['y'] +# => 3 +# +# # WRITE +# irb(main):002:0> line = map2dkvpline(map, '=', ',') +# => "x=1,y=2,z=3" +# +# ================================================================ + +# ---------------------------------------------------------------- +# ips and ifs (input pair separator and input field separator) are nominally '=' and ','. +def dkvpline2map(line, ips, ifs) + map = {} + line.split(ifs).each do |pair| + (k, v) = pair.split(ips, 2) + + # Type inference: + begin + v = Integer(v) + rescue ArgumentError + begin + v = Float(v) + rescue ArgumentError + # Leave as string + end + end + + map[k] = v + end + map +end + +# ---------------------------------------------------------------- +# ops and ofs (output pair separator and output field separator) are nominally '=' and ','. +def map2dkvpline(map, ops, ofs) + map.collect{|k,v| k.to_s + ops + v.to_s}.join(ofs) +end diff --git a/doc/polyglot-dkvp-io/example.py b/doc/polyglot-dkvp-io/example.py new file mode 100644 index 000000000..7b7f79eb2 --- /dev/null +++ b/doc/polyglot-dkvp-io/example.py @@ -0,0 +1,31 @@ +#!/usr/bin/env ruby + +import sys +import re +import dkvp_io + +while True: + # Read the original record: + line = sys.stdin.readline().strip() + if line == '': + break + map = dkvp_io.dkvpline2map(line, '=', ',') + + # Drop a field: + map.pop('x') + + # Compute some new fields: + map['ab'] = map['a'] + map['b'] + map['iy'] = map['i'] + map['y'] + + # Add new fields which show type of each already-existing field: + keys = map.keys() + for key in keys: + # Convert "" to just "int", etc.: + type_string = str(map[key].__class__) + type_string = re.sub("", "", type_string) + map['t'+key] = type_string + + # Write the modified record: + print dkvp_io.map2dkvpline(map, '=', ',') diff --git a/doc/polyglot-dkvp-io/example.rb b/doc/polyglot-dkvp-io/example.rb new file mode 100644 index 000000000..f1646e117 --- /dev/null +++ b/doc/polyglot-dkvp-io/example.rb @@ -0,0 +1,24 @@ +#!/usr/bin/env ruby + +require 'dkvp_io' + +ARGF.each do |line| + # Read the original record: + map = dkvpline2map(line.chomp, '=', ',') + + # Drop a field: + map.delete('x') + + # Compute some new fields: + map['ab'] = map['a'] + map['b'] + map['iy'] = map['i'] + map['y'] + + # Add new fields which show type of each already-existing field: + keys = map.keys + keys.each do |key| + map['t'+key] = map[key].class + end + + # Write the modified record: + puts map2dkvpline(map, '=', ',') +end diff --git a/doc/record-heterogeneity.html b/doc/record-heterogeneity.html index e3847dacd..62040072e 100644 --- a/doc/record-heterogeneity.html +++ b/doc/record-heterogeneity.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/reference-dsl.html b/doc/reference-dsl.html index 765a8b186..05a821fc4 100644 --- a/doc/reference-dsl.html +++ b/doc/reference-dsl.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/reference-verbs.html b/doc/reference-verbs.html index 45d381ea1..1933eecbb 100644 --- a/doc/reference-verbs.html +++ b/doc/reference-verbs.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/reference.html b/doc/reference.html index d7bee446e..c55bec2da 100644 --- a/doc/reference.html +++ b/doc/reference.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/release-docs.html b/doc/release-docs.html index a34557154..9e9cd0bca 100644 --- a/doc/release-docs.html +++ b/doc/release-docs.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/to-do.html b/doc/to-do.html index 5e81b0cfb..27135732c 100644 --- a/doc/to-do.html +++ b/doc/to-do.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/why.html b/doc/why.html index 5a32fb9bb..c58dea49e 100644 --- a/doc/why.html +++ b/doc/why.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3 diff --git a/doc/whyc.html b/doc/whyc.html index 316045ab9..34e33e41d 100644 --- a/doc/whyc.html +++ b/doc/whyc.html @@ -113,6 +113,7 @@ Miller commands were run with pretty-print-tabular output format.
• Internationalization
Using Miller:
• FAQ +
• Sharing data with other languages
• Cookbook part 1
• Cookbook part 2
• Cookbook part 3