LAVA Overview¶
What is LAVA?¶
LAVA is the Linaro Automation and Validation Architecture.
LAVA is a continuous integration system for deploying operating systems onto physical and virtual hardware for running tests. Tests can be simple boot testing, bootloader testing and system level testing, although extra hardware may be required for some system tests. Results are tracked over time and data can be exported for further analysis.
LAVA is a collection of participating components, the overall idea and evolving architecture that allows us to make testing, quality control and automation. LAVA-the-stack aims to make systematic, automatic and manual quality control more approachable for projects of all sizes.
LAVA is for validation - telling whether the code the other Linaro engineers are producing “works” in whatever sense that means. It could be a simple compile or boot test for the kernel, testing whether the code produced by gcc is smaller or faster, whether a kernel scheduler change reduces power consumption for a certain workload, or many other things.
Beyond simple validation though, what LAVA really about is automated validation. LAVA builds and tests the kernel on all supported boards every day. LAVA builds and tests proposed android changes in gerrit before they are landed, and the same for the gcc work. There is a validation lab in Cambridge - the boards from the Linaro members we want to test on, but also Cyclades serial console servers, routers, and a few servers.
Note
This overview document explains LAVA using
http://validation.linaro.org/ which is the official
production instance of LAVA hosted by Linaro. Where examples
reference validation.linaro.org
, replace with the fully
qualified domain name of your LAVA instance.
Logging In¶
Each LAVA instance offers a Local User Account and can also be configured to offer an alternative authentication method:
OpenID is no longer available for all installs of LAVA. The OpenID support changed in the 2015.12 release due to incompatible changes in the underlying django support. See User authentication for more information.
LDAP authentication relies on local configuration and may involve obtaining a privileged access token from the LDAP administrators.
Local accounts remain available for all instances.
Whichever authentication method is used, there is no difference in how users interact with LAVA, once logged in.
Local User Account¶
A local user account can be created by an administrator within LAVA. The administrator grants different permissions which allows the user to perform various operations within LAVA. A typical login screen will look like the following. In order to login with an username and password enter your credentials in the provided text boxes and click “Sign in with username and password” button
Local user accounts can be particularly useful for automated users like build systems or continuous integration scripts using the XMLRPC API. The local admin can create the user with a secure password, login as that user to create a token and then supply the token to the scripts.

Logging into the validation lab in Cambridge¶
The validation lab in Cambridge is accessible via http://validation.linaro.org/ which is the official production instance of LAVA.
The validation lab in Cambridge is now accessed using Linaro LDAP. In order to login with LDAP, enter your details in the LDAP login fields.
Note
If your Linaro email is first.second@linaro.org
then use
first.second
as your username.

A successful authentication will redirect you to http://validation.linaro.org/
If you do not have a Linaro LDAP account, you can Register with Linaro as a Community contributor.
The LAVA scheduler page looks like the following,

Authentication Tokens¶
In order to securely access LAVA resources via XML-RPC APIs using lava-tool or XML-RPC clients, an Authentication token should be created by the user.
Once the user is logged in to a LAVA instance (e.g. http://validation.linaro.org/) click on the “API” link which will bring up a drop down menu on which click on “Authentication Tokens” link. The following page will appear

Click on “new” button on the above page which raises a dialog box asking for a “Description” of the token to help you identify and delete tokens - if required at a later stage.

Click on the “Create” button to generate the token. The token will be assigned a token number (which is autogenerated by LAVA).

To display the token that was generated above click on the green “display this token” button in the Actions column on the above page.

If a token is compromised it can be removed by clicking on the red “delete this token” button in the Actions column. Anyone using that token will no longer be able to authenticate as You in the system.
lava-tool overview¶
lava-tool
is the command-line tool for interacting with the various
services offered by LAVA via XML-RPC APIs. The full list of API calls
is visible on the Available methods link from the API menu:
http://localhost/api/help
lava-tool
is primarily designed to assist users and uses desktop
integration hooks provided by python-keyring
and gnome-keyring
.
When scripts need to interact with LAVA, the XMLRPC API calls should be
used directly so that the scripts do not need to prompt for a password
to the local user keyring. Scripts used by build servers and continuous
integration tools should use a dedicated user created by the
administrator of a particular instance),
The API help page includes an example python script to connect to the local instance. To add token support, use the syntax username:token for the server concerned:
server = xmlrpclib.ServerProxy("https://%s:%s@%s/RPC2" % (username, token, server))
See XML-RPC for more information.
Installing lava-tool¶
lava-tool
is installed alongside LAVA by default, when the top
level lava
package is installed on a Debian-based distributions.
lava-tool
can also be installed on any remote machine running a
Debian-based distribution, without needing the rest of LAVA, to allow
a remote user to interact with any LAVA instance on which the user has
an account.:
$ sudo apt-get update
$ sudo apt-get install lava-tool
Using lava-tool¶
Once the token is created add it to lava-tool as follows. Click on “display the token” link on the “Authentication Tokens” page and copy the token. e.g. if your token was created on validation.linaro.org:
$ lava-tool auth-add http://<username>@validation.linaro.org/RPC2/
Paste token for http://<username>@validation.linaro.org/RPC2/:
Please set a password for your new keyring:
Please confirm the password:
Token added successfully for user <username>.
Note
Paste the token copied previously when it is asked above. Replace username with your username. If the keyring is created newly, set a password for the same, else enter the password for the keyring already created.
Bundle Stream Overview¶
What is a Bundle Stream?¶
LAVA runs tests which produce results with multiple tests being run for each submitted job. The collection of results from any one submitted job is termed a Result Bundle. Each bundle can contain multiple sets of test results, as well as other information about the system where the testing was performed.
Within a single result bundle are the results of each test definition execution, termed a Test Run. Each Test Run is typically a single YAML file and is listed in the bundle via the description of the test definition. The individual id and result of a single test within a test run is called the Test Case, typically a single line in the YAML file. If the job ran across multiple devices, the bundle can include test runs from each device from that job.
Result Bundles are uploaded to the server at the end of the test run into a Bundle Stream which is a way of organising related results bundles. A bundle stream could be imagined as a folder within which all related result bundle will be stored. A bundle stream could be private or anonymous. The name of the stream is specified in the job definition to determine where the result bundle from the job should be submitted.
How to setup a Bundle Stream?¶
A public/anonymous bundle stream could be setup with the help of lava-tool as follows,
$ lava-tool make-stream --dashboard-url
http://<username>@validation.linaro.org/RPC2/ /anonymous/USERNAME/
Note
Replace username and USERNAME with your username. Alternatively an existing stream like anonymous/test could be used for initial testing purposes.
Submitting your first job¶
A job defines what image to deploy on the DUT and further actions that should be performed on the DUT. Jobs are defined in JSON files.
Job Definition¶
Here’s a minimal job that could be executed
{
"job_name": "kvm-test",
"device_type": "kvm",
"timeout": 1800,
"actions": [
{
"command": "deploy_linaro_image",
"parameters":
{
"image": "http://images.validation.linaro.org/kvm-debian-wheezy.img.gz"
}
},
{
"command": "boot_linaro_image"
},
{
"command": "submit_results",
"parameters":
{
"server": "http://<username>@validation.linaro.org/RPC2/",
"stream": "/anonymous/test/"
}
}
]
}
Note
Replace username with your username.
Job Submission¶
A job could be submitted either from the command line (using lava-tool) or via the web UI.
Command Line Job Submission¶
Once you have copied the above job definition to a file, for example /tmp/job.json Use the lava-tool as shown below,
$ lava-tool submit-job https://<username>@validation.linaro.org/RPC2/
/tmp/job.json
Please enter password for encrypted keyring:
submitted as job id: 82287
Note
Replace username with your username. Enter the password for the encrypted keyring which is the same that was used when adding authentication token.
Once the job is submitted successfully, the job-id is returned back,
which could be used in order to check the status of the job on the
UI. In the above submission job-id returned is 82287. Visit
http://validation.linaro.org/scheduler/job/<job-id>
in order to see
the details of the job run.
Job Definition¶
Here’s a minimal job that could be executed
{
"job_name": "kvm-test",
"device_type": "kvm",
"timeout": 1800,
"actions": [
{
"command": "deploy_linaro_image",
"parameters":
{
"image": "http://images.validation.linaro.org/kvm-debian-wheezy.img.gz"
}
},
{
"command": "boot_linaro_image"
},
{
"command": "submit_results",
"parameters":
{
"server": "http://<username>@validation.linaro.org/RPC2/",
"stream": "/anonymous/test/"
}
}
]
}
Note
Replace username with your username.
Job Submission¶
A job could be submitted either from the command line (using lava-tool) or via the web UI.
Command Line Job Submission¶
Once you have copied the above job definition to a file, for example /tmp/job.json Use the lava-tool as shown below,
$ lava-tool submit-job https://<username>@validation.linaro.org/RPC2/
/tmp/job.json
Please enter password for encrypted keyring:
submitted as job id: 82287
Note
Replace username with your username. Enter the password for the encrypted keyring which is the same that was used when adding authentication token.
Once the job is submitted successfully, the job-id is returned back,
which could be used in order to check the status of the job on the
UI. In the above submission job-id returned is 82287. Visit
http://validation.linaro.org/scheduler/job/<job-id>
in order to see
the details of the job run.
Web Based Job Submission¶
Visit http://validation.linaro.org/scheduler/jobsubmit and paste your json file into the window and click “Submit” button. The job submission screen is shown below,

Note
If a link to job json file is pasted on the above screen, the JSON file will be fetched and displayed in the text box for submission.
Once the job is successfully submitted, the following screen appears, from which the user can navigate to the job details or the list of jobs page.

Viewing the submitted job will show something like this.

Test Jobs¶
In order to run a test, a test definition is required. A test definition is expressed in YAML format. A minimal test definition would look like the following
metadata:
name: passfail
format: "Lava-Test-Shell Test Definition 1.0"
description: "Pass/Fail test."
version: 1.0
run:
steps:
- "lava-test-case passtest --result pass"
- "lava-test-case failtest --result pass"
In order to run the above test definition with a minimal job file, the following job json could be used and submitted in the same way as explained above
run:
steps:
- "lava-test-case passtest --result pass"
- "lava-test-case failtest --result pass"
In order to run the above test definition with a minimal job file, the following job json could be used and submitted in the same way as explained above
{
"job_name": "kvm-test",
"device_type": "kvm",
"timeout": 1800,
"actions": [
{
"command": "deploy_linaro_image",
"parameters": {
"image":
"http://images.validation.linaro.org/kvm-debian-wheezy.img.gz"
}
},
{
"command": "lava_test_shell",
"parameters": {
"testdef_urls": [
"http://people.linaro.org/~senthil.kumaran/test.yaml"
]
}
},
{
"command": "boot_linaro_image"
},
{
"command": "submit_results",
"parameters": {
"server":
"http://stylesen@validation.linaro.org/RPC2/",
"stream": "/anonymous/test/"
}
}
]
}
Note
The test definition is uploaded to an URL that will be accessible over http which is referred in the job json.
Note
Test definitions could be referred from git repositories. The official upstream Linaro git repository for test definitions is https://git.linaro.org/gitweb?p=qa/test-definitions.git
Using tables in LAVA¶
LAVA presents much of the data about jobs, devices, results and tasks inside tables. The length of a table can be controlled, the table can be sorted by selected columns and the data itself can be searched. All options can be controlled from the query string in the browser address bar. This allows particular views of a table to be shared as links. See Custom table queries.
For pages which only contain a single table, the number of rows displayed in each page of data is controlled via the length parameter. For convenience, there is a drop down box on the left of each table where the table length can be selected.
Table search support¶
Unless specified explicitly, all searches are case-sensitive.
Simple text search¶
The search box above each table allows arbitrary text strings to be used as filters on the data within the table. Each table has support for matching simple text strings against certain columns within the table and these searches are additive - the data in the row will be included in the results if the text matches any of the search fields.
The fields which support text search are listed above each table.
Custom table queries¶
Some tables also support customised queries on specific fields, typically these will be time based fields like submit_time, end_time and duration. These queries allow a specific function to be called within the filter to match only results where the timestamp occurred within the specified number of minutes, hours or days, relative to the current time on the server.
The queries supported by a table are listed above the table, along with details of whether that query is based on minutes, hours or days.
Note
Time based queries will always take the current time on the server into account, so links containing such queries may not give the same results when viewed at a later time.
Time based queries can take calculations in the query string as well,
e.g. end_time
is based on hours, so ?end_time=4*24
matches
end_time
within the last 4 days (the search summary will still show
the 4*24
.)
Exclusive table searches¶
Fields used in simple text searches can also be used as exclusive searches by adding the exclusive search field to the querystring. The data in the row will be included in the results only if the text matches all of the search fields:
?device=mx5&length=25&description=ARMMP&status=Incom
This querystring would only show rows where the device hostname contains
mx5
and the description contains ARMMP
and the status of
the job contains Incom
, therefore showing up to 25 results for jobs
on such devices with that description which finished with a status of
Incomplete.
Note
Exclusive searches are not supported via the search box in the table header. Add to the querystring directly. Exclusive text search cannot be combined with simple text search. Replace the search variable in the querystring with the closest discrete query term, e.g. description.
The fields which support exclusive search are listed above each table.
Other filters¶
Individual tables may also provide filters via javascript or other support.
Resetting a table¶
The breadcrumb link should take you back to the default table. Alternatively, clear the querystring in the browser address bar manually.