Option handling in Tradefed

Option handling lies at the heart of Trade Federation's modular approach. In particular, options are the mechanism by which the developer, Integrator, and Test Runner can work together without having to duplicate each-other's work. Put simply, our implementation of option handling allows the developer to mark a Java class member as being configurable, at which point the value of that member can be augmented or overridden by the Integrator, and can be subsequently augmented or overridden by the Test Runner. This mechanism works for all Java intrinsic types, as well as for any Map or Collection instances of intrinsic types.

Note: The option-handling mechanism only works for classes implementing one of the interfaces included in the Test Lifecycle, and only when that class is instantiated by the lifecycle machinery.

Developer

To start off, the developer marks a member with the @Option annotation. They specify (at a minimum) the name and description values, which specify the argument name associated with that Option, and the description that is displayed on the TF console when the command is run with --help or --help-all.

As an example, let's say we want to build a functional phone test that dials a variety of phone numbers, and expects to receive a sequence of DTMF tones from each number after it connects.

public class PhoneCallFuncTest extends IRemoteTest {
    @Option(name = "timeout", description = "How long to wait for connection, in millis")
    private long mWaitTime = 30 * 1000;  // 30 seconds

    @Option(name = "call", description = "Key: Phone number to attempt. " +
            "Value: DTMF to expect. May be repeated.")
    private Map<String, String> mCalls = new HashMap<String, String>;

    public PhoneCallFuncTest() {
        mCalls.add("123-456-7890", "01134");  // default
    }

That's all that's required for the developer to set up two points of configuration for that test. They could then go off and use mWaitTime and mCalls as normal, without paying much attention to the fact that they're configurable. Because the @Option fields are set after the class is instantiated, but before the run method is called, that provides an easy way for implementors to set up defaults for or perform some kind of filtering on Map and Collection fields, which are otherwise append-only.

Integrator

The Integrator works in the world of configurations, which are written in XML. The config format allows the Integrator to set (or append) a value for any @Option field. For instance, suppose the Integrator wanted to define a lower-latency test that calls the default number, as well as a long-running test that calls a variety of numbers. They could create a pair of configurations that might look like the following:

<?xml version="1.0" encoding="utf-8"?>
<configuration description="low-latency default test; low-latency.xml">
    <test class="com.example.PhoneCallFuncTest">
        <option name="timeout" value="5000" />
    </test>
</configuration>

<?xml version="1.0" encoding="utf-8"?>
<configuration description="call a bunch of numbers; many-numbers.xml">
    <test class="com.example.PhoneCallFuncTest">
        <option name="call" key="111-111-1111" value="#*#*TEST1*#*#" />
        <option name="call" key="222-222-2222" value="#*#*TEST2*#*#" />
        <!-- ... -->
    </test>
</configuration>

Test Runner

The Test Runner also has access to these configuration points through the Trade Federation console. First and foremost, they run a command (that is, a config and all of its arguments) with the run command <name> instruction (or run <name> for short). Beyond that, they can specify any list of arguments are part of the command, which can replace or append to fields specified by lifecycle objects within each config.

To run the low-latency test with the many-numbers phone numbers, the Test Runner could execute:

tf> run low-latency.xml --call 111-111-1111 #*#*TEST1*#*# --call 222-222-2222 #*#*TEST2*#*#

Or, to get a similar effect from the opposite direction, the Test Runner could reduce the wait time for the many-numbers test:

tf> run many-numbers.xml --timeout 5000

Option ordering

You might notice that the call option underlying implementation is a Map so upon repeated --call on the command line, they're all stored.

The option timeout, which has an underlying implementation of long, can only store one value. So only the last value specified is stored. --timeout 5 --timeout 10 results in timeout containing 10.

In case of a List or Collection as the underlying implementation, all the values are stored, in the order specified on the command line.

Boolean options

Options of boolean underlying type can be set to true by directly passing the option name, for example, --[option-name] and can be set to false using the syntax --no-[option-name].

See also