CS 330: Lecture 37 – Object-relational Mapping Via Metaprogramming
Dear students,
Metaprogramming is our theme for this last week of the semester. What is metaprogramming? It’s when code generates code. It’s when user input is used to generate new classes, methods, and so on. You’ll probably all felt at certain points that the code you are writing probably didn’t need to be written by a human. Perhaps its repetitive. Perhaps its a direct translation of some other structure found elsewhere in your technology stack.
This class has largely been about how we as developers model ideas about computation using programming languages. We’ve mostly ignored the end user and focused on how we can make programming more enjoyable, more efficient, and more intellectually stimulating. Metaprogramming certainly fulfills all these goals.
We will examine metaprogramming in both Ruby and Java. Today, we will concentrate on Ruby. In particular, we will construct a lightweight system for object-relational mapping. In an ORM, you’ve got a representation of some data type in one system, and you’d like to use that same type in another system. Certainly you could specify the type twice, but that’s prone to error. The two specifications will inevitably become unsynchronized.
ORM is often used with databases and the languages that access them. You might write a schema to describe a table in SQL, and you want the code to automatically understand the schema without you having to do any extra work. Here are some things that our ORM should be able to handle:
- deserializing
- serializing
- reading and writing of properties, including properties that are themselves objects
A full ORM will support more features, like relations to other records. Our database will not be a database but a simpler JSON file:
{
"headline": "Americans Scroll 2.5 Miles Per Day",
"nwords": 849,
"author": "Petey F.",
"copy": "...",
"tags": ["internet", "millenials"]
}
Out of the box, we can get reasonably close to turning this text into a Ruby object simply by parsing it using the JSON API:
json = File.read(ARGV[0])
article = JSON.parse(json, symbolize_names: true)
puts article[:author]
But this isn’t really an object. It’s a dictionary/hash/key-value pair manager. It’d be sweet if we could make Ruby more like Javascript, where there’s an equivalence between dictionary lookup and field access:
var foo = {};
foo['name'] = 'Scout';
console.log(foo.name);
We’d like to make our Ruby ORM build structures that feel more like an object. Instead of saying this:
article[:author] = value
we’d like to be able to say this:
article.author = value
Let’s do it. We’ll need a new class to support this:
class Autobject
# ...
end
What happens currently when we try to read a property of Autobject?
article = Autobject.new
puts article.author
When we run this code, we see that method author cannot be found. Is there any way that we can write a class that has methods for every field/property that our clients may want to assign? No way. We can’t see that far into the future.
For this to happen, we need some metaprogramming. We need to generate these methods on the fly based on our schema. But first, let’s write a minimal constructor for testing out some ideas:
def initialize value = {}
@data = value
end
If the client provides no value, we’ll just wrap around an empty dictionary, a blank data store.
Now, how do we handle all these infinite methods that are impossible to write? Easy. A catch-all method that Ruby will call on our objects when they don’t support a method. It’s called method_missing:
def method_missing symbol, *args
...
end
What might be true when this method is called?
- It might be a read operation for a key already in the dictionary.
- It might be a read operation for a key not already in the dictionary.
- It might be a write operation.
Let’s handle the first case:
if @data.has_key? symbol
@data[symbol]
end
If the key doesn’t exist, let’s raise an exception:
else
raise "No such property: #{symbol.to_s}"
end
The last case is a bit more involved. We have to check if the method name suggests an assignment, but this needs to be done on the string version of the symbol:
elsif symbol.to_s =~ /^(\w+)=$/ && args.length == 1
@data[$1.to_sym] = args[0]
end
Okay, let’s test this out:
f = Autobject.new first: 'Roy', last: 'Biv'
f.middle = 'G'
puts "#{f.first} #{f.middle} #{f.last}"
Now let’s get this to work with JSON data. Let’s add a static method for loading an Autobject from some other source:
def self.load src
...
end
If we have a URI or File, we’ll open it and slurp up the JSON contents. Otherwise, we’ll assume we have a JSON string. Once we know we have JSON, we can parse it:
if src.is_a?(URI) || src.is_a?(File)
json = open(src) do |io|
io.read
end
else
json = src
end
Autobject.new(JSON.parse(json, symbolize_names: true))
Now, let’s try reading some JSON:
g = Autobject.load '{"first": "Roy", "last": "Biv"}'
puts g.inspect
And some weather data:
weather = Autobject.load(URI("http://api.openweathermap.org/data/2.5/weather?q=Eau%20Claire%2CWisconsin&APPID=#{KEY}"))
puts weather.inspect
Let’s try to pull out the current temperature:
puts weather.main.temp
This fails. Why? weather.main gives back a Hash. Calling temp on a Hash is bound to fail. We want main to also be an Autobject so we can access fields inside of it with the . operator. Let’s write a little recursive helper that goes and turns all the parts of our JSON structure into Autobjects:
def self.objectify data
if data.is_a? Array
data.map do |element|
Autobject.objectify(element)
end
elsif data.is_a? Hash
object = data.transform_values do |value|
Autobject.objectify(value)
end
Autobject.new object
else
data
end
end
And then in load, we can apply this method to the data that we parse:
data = JSON.parse(json, symbolize_names: true)
Autobject.objectify(data)
Our temperature reading should work now! How about the time of the sunset?
puts Time.at(weather.sys.sunset)
What’s really nice about metaprogramming is that the code is essentially writing itself. This Autobject should be able to handle any key and any dictionary-ish object. Let’s try getting the high and low temperatures:
weather = Autobject.load(URI("http://api.openweathermap.org/data/2.5/forecast/daily?units=imperial&q=Eau%20Claire%2CWisconsin&APPID=#{KEY}"))
puts weather.list[0].temp.min
puts weather.list[0].temp.max
Let’s try dumping the object en masse to the console:
def inspect
JSON.pretty_generate(@data)
end
This seems to only work for the top-level dictionary. The pretty_generate method recursively calls to_json on all the values nested inside our object, so we also need to provide that method:
def to_json generator
JSON.pretty_generate(@data, generator)
end
Of course, now that we have the ability to print an object, we should provide a method for dumping it out to disk:
def save path
open(path, 'w') do |file|
file.puts inspect
end
end
Now we have automatic getters, setters, serialization, and deserialization for any sort of object we can dream up. Thanks, metaprogramming.
See you next time!
P.S. It’s time for a haiku!
Taught my car to drive
Now when we go on roadtrips
Two can ride shotgun
P.P.S. Here’s the code we wrote together!
article.json
{
"headline": "Americans Scroll 2.5 Miles Per Day",
"nwords": 849,
"author": "Petey F.",
"copy": "...",
"tags": ["internet", "millenials"]
}
meta.rb
#!/usr/bin/env ruby
require 'json'
require 'open-uri'
json = File.read(ARGV[0])
article = JSON.parse(json)
# puts article
# puts "article.class: #{article.class}"
class Auto
def initialize data = {}
@data = data
end
def method_missing symbol, *args
if @data.has_key?(symbol)
@data[symbol]
elsif symbol[-1] == '='
@data[symbol.to_s.chop.to_sym] = args[0]
else
raise "No such property #{symbol}... I have failed you. No, it's me."
end
end
def self.load(src)
if src.is_a?(File) || src.is_a?(URI)
json = open(src) do |io|
io.read
end
else
json = src
end
data = JSON.parse(json, symbolize_names: true)
Auto.objectify(data)
end
def inspect
JSON.pretty_generate(@data)
end
def self.objectify data
if data.is_a? Array
data.map do |element|
Auto.objectify(element)
end
elsif data.is_a? Hash
object = data.transform_values do |value|
Auto.objectify(value)
end
Auto.new object
else
data
end
end
end
article = Auto.load(File.new(ARGV[0]))
puts "article.author: #{article.author}"
puts "article.nwords: #{article.nwords}"
puts "article.headline: #{article.headline}"
# article = Auto.new
# article.author = "H.G. Wells"
# p article
# puts article.author
# puts article['headline']
# puts article.headline
KEY = 'INSERT-YOUR-KEY-HERE'
weather = Auto.load(URI("http://api.openweathermap.org/data/2.5/weather?q=Eau%20Claire%2CWisconsin&APPID=#{KEY}"))
p Time.at(weather.sys.sunset)
# puts weather.main.temp
# p weather