» CS 330: Lecture 37 – Object-relational Mapping Via Metaprogramming

CS 330: Lecture 37 – Object-relational Mapping Via Metaprogramming

May 7, 2018 by Chris Johnson. Filed under cs330, lectures, spring 2018.

Dear students,

Metaprogramming is our theme for this last week of the semester. What is metaprogramming? It’s when code generates code. It’s when user input is used to generate new classes, methods, and so on. You’ll probably all felt at certain points that the code you are writing probably didn’t need to be written by a human. Perhaps its repetitive. Perhaps its a direct translation of some other structure found elsewhere in your technology stack.

This class has largely been about how we as developers model ideas about computation using programming languages. We’ve mostly ignored the end user and focused on how we can make programming more enjoyable, more efficient, and more intellectually stimulating. Metaprogramming certainly fulfills all these goals.

We will examine metaprogramming in both Ruby and Java. Today, we will concentrate on Ruby. In particular, we will construct a lightweight system for object-relational mapping. In an ORM, you’ve got a representation of some data type in one system, and you’d like to use that same type in another system. Certainly you could specify the type twice, but that’s prone to error. The two specifications will inevitably become unsynchronized.

ORM is often used with databases and the languages that access them. You might write a schema to describe a table in SQL, and you want the code to automatically understand the schema without you having to do any extra work. Here are some things that our ORM should be able to handle:

deserializing
serializing
reading and writing of properties, including properties that are themselves objects

A full ORM will support more features, like relations to other records. Our database will not be a database but a simpler JSON file:

{
  "headline": "Americans Scroll 2.5 Miles Per Day",
  "nwords": 849,
  "author": "Petey F.",
  "copy": "...",
  "tags": ["internet", "millenials"]
}

Out of the box, we can get reasonably close to turning this text into a Ruby object simply by parsing it using the JSON API:

json = File.read(ARGV[0])
article = JSON.parse(json, symbolize_names: true)
puts article[:author]

But this isn’t really an object. It’s a dictionary/hash/key-value pair manager. It’d be sweet if we could make Ruby more like Javascript, where there’s an equivalence between dictionary lookup and field access:

var foo = {};
foo['name'] = 'Scout';
console.log(foo.name);

We’d like to make our Ruby ORM build structures that feel more like an object. Instead of saying this:

article[:author] = value

we’d like to be able to say this:

article.author = value

Let’s do it. We’ll need a new class to support this:

class Autobject
  # ...
end

What happens currently when we try to read a property of Autobject?

article = Autobject.new
puts article.author

When we run this code, we see that method author cannot be found. Is there any way that we can write a class that has methods for every field/property that our clients may want to assign? No way. We can’t see that far into the future.

For this to happen, we need some metaprogramming. We need to generate these methods on the fly based on our schema. But first, let’s write a minimal constructor for testing out some ideas:

def initialize value = {}
  @data = value
end

If the client provides no value, we’ll just wrap around an empty dictionary, a blank data store.

Now, how do we handle all these infinite methods that are impossible to write? Easy. A catch-all method that Ruby will call on our objects when they don’t support a method. It’s called method_missing:

def method_missing symbol, *args
  ...
end

What might be true when this method is called?

It might be a read operation for a key already in the dictionary.
It might be a read operation for a key not already in the dictionary.
It might be a write operation.

Let’s handle the first case:

if @data.has_key? symbol
  @data[symbol]
end

If the key doesn’t exist, let’s raise an exception:

else
  raise "No such property: #{symbol.to_s}"
end

The last case is a bit more involved. We have to check if the method name suggests an assignment, but this needs to be done on the string version of the symbol:

elsif symbol.to_s =~ /^(\w+)=$/ && args.length == 1
  @data[$1.to_sym] = args[0]
end

Okay, let’s test this out:

f = Autobject.new first: 'Roy', last: 'Biv'
f.middle = 'G'
puts "#{f.first} #{f.middle} #{f.last}"

Now let’s get this to work with JSON data. Let’s add a static method for loading an Autobject from some other source:

def self.load src
  ...
end

If we have a URI or File, we’ll open it and slurp up the JSON contents. Otherwise, we’ll assume we have a JSON string. Once we know we have JSON, we can parse it:

if src.is_a?(URI) || src.is_a?(File)
  json = open(src) do |io|
    io.read
  end
else
  json = src
end

Autobject.new(JSON.parse(json, symbolize_names: true))

Now, let’s try reading some JSON:

g = Autobject.load '{"first": "Roy", "last": "Biv"}'
puts g.inspect

And some weather data:

weather = Autobject.load(URI("http://api.openweathermap.org/data/2.5/weather?q=Eau%20Claire%2CWisconsin&APPID=#{KEY}"))
puts weather.inspect

Let’s try to pull out the current temperature:

puts weather.main.temp

This fails. Why? weather.main gives back a Hash. Calling temp on a Hash is bound to fail. We want main to also be an Autobject so we can access fields inside of it with the . operator. Let’s write a little recursive helper that goes and turns all the parts of our JSON structure into Autobjects:

def self.objectify data
  if data.is_a? Array
    data.map do |element|
      Autobject.objectify(element)
    end
  elsif data.is_a? Hash
    object = data.transform_values do |value|
      Autobject.objectify(value)
    end
    Autobject.new object
  else
    data
  end
end

And then in load, we can apply this method to the data that we parse:

data = JSON.parse(json, symbolize_names: true)
Autobject.objectify(data)

Our temperature reading should work now! How about the time of the sunset?

puts Time.at(weather.sys.sunset)

What’s really nice about metaprogramming is that the code is essentially writing itself. This Autobject should be able to handle any key and any dictionary-ish object. Let’s try getting the high and low temperatures:

weather = Autobject.load(URI("http://api.openweathermap.org/data/2.5/forecast/daily?units=imperial&q=Eau%20Claire%2CWisconsin&APPID=#{KEY}"))
puts weather.list[0].temp.min
puts weather.list[0].temp.max

Let’s try dumping the object en masse to the console:

def inspect
  JSON.pretty_generate(@data)
end

This seems to only work for the top-level dictionary. The pretty_generate method recursively calls to_json on all the values nested inside our object, so we also need to provide that method:

def to_json generator
  JSON.pretty_generate(@data, generator)
end

Of course, now that we have the ability to print an object, we should provide a method for dumping it out to disk:

def save path
  open(path, 'w') do |file|
    file.puts inspect
  end
end

Now we have automatic getters, setters, serialization, and deserialization for any sort of object we can dream up. Thanks, metaprogramming.

See you next time!

Sincerely,

P.S. It’s time for a haiku!

Taught my car to drive
Now when we go on roadtrips
Two can ride shotgun

P.P.S. Here’s the code we wrote together!

article.json

{
  "headline": "Americans Scroll 2.5 Miles Per Day",
  "nwords": 849,
  "author": "Petey F.",
  "copy": "...",
  "tags": ["internet", "millenials"]
}

meta.rb

#!/usr/bin/env ruby

require 'json'
require 'open-uri'

json = File.read(ARGV[0])
article = JSON.parse(json)

# puts article
# puts "article.class: #{article.class}"

class Auto
  def initialize data = {}
    @data = data
  end

  def method_missing symbol, *args
    if @data.has_key?(symbol)
      @data[symbol]
    elsif symbol[-1] == '='
      @data[symbol.to_s.chop.to_sym] = args[0]
    else
      raise "No such property #{symbol}... I have failed you. No, it's me."
    end
  end

  def self.load(src)
    if src.is_a?(File) || src.is_a?(URI)
      json = open(src) do |io|
        io.read
      end
    else
      json = src
    end

    data = JSON.parse(json, symbolize_names: true)
    Auto.objectify(data)
  end

  def inspect
    JSON.pretty_generate(@data)
  end

  def self.objectify data
    if data.is_a? Array
      data.map do |element|
        Auto.objectify(element)
      end
    elsif data.is_a? Hash
      object = data.transform_values do |value|
        Auto.objectify(value)
      end
      Auto.new object
    else
      data
    end
  end
end

article = Auto.load(File.new(ARGV[0]))
puts "article.author: #{article.author}"
puts "article.nwords: #{article.nwords}"
puts "article.headline: #{article.headline}"

# article = Auto.new
# article.author = "H.G. Wells"
# p article
# puts article.author

# puts article['headline']
# puts article.headline

KEY = 'INSERT-YOUR-KEY-HERE'
weather = Auto.load(URI("http://api.openweathermap.org/data/2.5/weather?q=Eau%20Claire%2CWisconsin&APPID=#{KEY}"))
p Time.at(weather.sys.sunset)

# puts weather.main.temp
# p weather