» SpecCheck

SpecCheck

July 13, 2012 by Chris Johnson. Filed under projects, public.

SpecCheck is a system for computer science educators teaching large classes to get their students’ code in line with homework specifications. With SpecCheck, educators write their reference implementation, tag the specified parts with @Specified, and generate tests for interface conformance that students apply to their own code. Wherever things are misnamed, wherever parameters are missing, wherever types are wrong, a failed SpecCheck test will let the students know—before the broken code reaches the grader.

The payoff of SpecCheck is that test-driven grading can run more smoothly. Deviations from the homework specification are minimized, so there’s less danger of linking problems between test code and student code.

For more discussion on its development and purpose, see my presentation at ITiCSE 2012 and the accompanying paper.

Invitation

If you are interested in doing a formal evaluation of SpecCheck’s impact, I’d welcome your collaboration. I’m not scheduled to teach an introductory course in the immediate future. Please drop me a line in the comments or by emailing me at johnch, domain uwec.edu.

HOWTO

Using SpecCheck for your classes is not a lot of work. Follow these few steps to get up and running.

1. Download

SpecCheck is released under the GNU Public License v3 and is freely available for download as an Eclipse project at GitHub. Choose Downloads to grab a complete archive.

I import the SpecCheck code as a standalone project in my Eclipse workspace. I add this project as a dependency to my other course projects by configuring their properties to include the SpecCheck project on their build path. Other IDEs support a similar setup.

2. Write your specification

Decide what you want your students to do. What classes must they write? What are the signatures of the classes’ constructors and methods?

For example:

Write a class Hero that has a constructor taking a String name as its sole parameter. Provide also a method getHitPoints that returns the Hero’s hit points as an int.

3. Create reference implementation

Implement your homework specification. Label any methods and fields required by the assignment with the @Specified annotation.

See org.twodee.speccheck.example.* for examples of tuning @Specified to check for things like supertypes and instance variable counts. Read the documentation in org.twodee.speccheck.Specified for a complete list of supported tests.

For example:

public class Hero {
  private String name;
  private int hitPoints;

  @Specified
  public Hero(String name) {
    this.name = name;
  }

  @Specified
  public int getHitPoints() {
    return hitPoints;
  }
}

3. Generate tests

Generate interface-conformance tests like so:

SpecCheck.generate(Hero.class, ...);

The generated tests will be printed to the console. Copy and paste these into a test suite file, perhaps named SpecChecker.java. Also in this file, add a little main method to run the tests and display the results in the console:

public static void main(String[] args) {
  SpecCheck.testAsStudent(SpecChecker.class);
}

I suggest this because without a main method, the class will likely be recognized as a JUnit test suite and, depending on your IDE, the results will appear in the IDE’s JUnit interface, which new programmers may find confusing to navigate. SpecCheck.testAsStudent simply prints the errors in the IDE’s console.

4. Distribute tests

Distribute the SpecChecker class and the entire org.twodee.speccheck package to your students. You can distribute the collection as a runnable JAR, with your SpecChecker as the main class. Instruct your students to import the test suite, add JUnit 4 to the build path, run the tests, and correct any non-conforming code.

teaching machines