Android 13 Camera Image Test Suite release notes

The Android 13 release includes a number of changes to Camera ITS. In addition to incremental changes such as updated Python and package versions and test hardware updates, Android 13 supports video testing.

This page summarizes the Camera ITS changes for Android 13. The changes fall into seven broad categories:

• Python and package versions
• Config file additions
• Test changes
• New tablet-based tests
• Video testing
• Hardware updates and additions
• Foldable testing support

Python and package versions

Android 13 support the following Python versions in addition to the Python versions and libraries supported in Android 12:

• Python 3.9.2
• OpenCV 4.2.0
• Numpy 1.20.3
• Matplotlib 3.3.4
• Scipy 1.6.2
• pySerial 3.5
• Pillow 8.3.1
• PyYAML 5.4.1
• Mobly 1.11
• FFmpeg 4.4.1

Config file additions

Due to the addition of lighting control for the test_auto_flash.py test, the config.yml file needs two additional parameters for the controller and lighting channel. To identify if the device under test (DUT) is a foldable or not, the config.yml file requires a third additional parameter that must be added to both the tablet-based and sensor fusion sections.

TestBeds:
  - Name: TEST_BED_TABLET_SCENES
    # Test configuration for scenes[0:4, 6, _change]
    Controllers:
        AndroidDevice:
          - serial: 8A9X0NS5Z
            label: dut
          - serial: 5B16001229
            label: tablet

    TestParams:
      brightness: 192
      chart_distance: 22.0
      debug_mode: "False"  # "True" or "False"; quotes needed
      lighting_cntl: <controller-type>  # "arduino" or "None"; quotes needed
      lighting_ch: <controller-channel>
      camera: 0
      foldable_device: "False". # set "True" if testing foldable
      scene: <scene-name>  # if <scene-name> runs all scenes

Test changes

scene1_1/test_black_white.py

The test_black_white test has channel saturation checking consistent with previous versions of Android, which means that the required first API level for the channel saturation check is Android 10.

scene1_2/test_yuv_plus_raw.py

The test_yuv_plus_raw test handles non 16:9 or 4:3 sensors to provide better support for RAW. If the test can't find a common format between the sensor RAW format and the YUV capture formats, the test compares the RAW capture to the largest YUV capture even if the aspect ratios are different.

scene2_a/test_faces.py

The test_faces is refactored into test_num_faces.

scene2_a/test_num_faces.py

The test_num_faces test handles sensor crop and places the face rectangles correctly on UW camera captures that are cropped.

scene3/test_lens_position.py

The test_lens_position test is deprecated in Android 13.

scene6/test_zoom.py

The test_zoom test is refactored to enable easier testing of three and four camera systems. If the camera tests correctly over a 10x ratio range in zoom, the test exits correctly. This enables zoom testing to be performed at a single chart distance.

scene_change/test_scene_change.py

The test_scene_change test is deprecated in Android 13.

New tablet-based tests

Android 13 includes two new tablet-based tests. Both tests are mandated for devices launching with Android 13, but are skipped for devices upgrading to Android 13.

scene2_a/test_auto_flash.py

Android 13 adds the test_auto_flash test.

Parameters

• flash: Skips test if flash isn't available.

Method

The test configures the capture request for auto-flash and presents a scene requiring a flash event. The lighting in the test rig and the tablet are turned off to present a dark scene for the auto-exposure algorithm. The test configures a sequence of NUM_FRAMES captures with auto-flash enabled. If the AE_STATE setting returns FLASH_REQUIRED the test verifies that a flash is fired in the captures.

Lighting can be controlled manually when prompted, or automatically with an Arduino controller with lighting control added for automated testing. A new utility folder lighting_control_utils centralizes code for controlling the lighting in tests.

scene2_a/test_yuv_jpeg_capture_sameness.py

Android 13 adds the test_yuv_jpeg_capture_sameness test.

Parameters

• streamUseCase: Determines whether the camera supports the stream use case.
• android.jpeg.quality: Sets JPEG quality to 100.

Method

This test captures two images using the largest common YUV and JPEG formats with the same aspect ratio as the largest JPEG format not exceeding a resolution of 1920x1440. The test sets jpeg.quality to 100 and captures a dual surface request. It then converts both images to RGB arrays and calculates the 3D root mean square (RMS) difference between the two images. The difference tolerance is set to 1%.

Video testing

Camera ITS adds support for video testing in Android 13.

Android 13 adds the following tests:

scene4/test_preview_stabilization_fov.py

Android 13 adds the test_preview_stabilization_fov test. This test checks the supported preview sizes to ensure the FoV isn't cropped inappropriately.

Method

The test captures two videos, one with preview stabilization on, and one with preview stabilization off. A representative frame is selected from each video, and analyzed to ensure that the FoV changes in the two videos are within specifications.

Specifically, the test checks for the following parameters with and without preview stabilization:

• The circle roundness remains constant.
• The center of the circle remains stable.
• The size of circle changes by no more that 20%, meaning that the FoV changes at most 20%.

scene4/test_video_aspect_ratio_and_crop.py

Android 13 adds the test_video_aspect_ratio_and_crop test. Similar to the test_aspect_ratio_and_crop test for still captures, this test checks the supported video formats to ensure the video frames aren't stretched or cropped inappropriately. All self-reported video qualites are tested. Additionally, if the camera supports HLG10 video, 10-bit video is tested.

Android 13 adds the get_available_video_qualities method to its_session_utils. To share common code with existing tests, Android 13 also includes two new utility functions, image_fov_utils and video_processing_utils.

Method

The test captures a reference image with the RAW format or the highest resolution JPEG format if RAW isn't supported. From the reference image, the circle size and location are determined. Short videos are then taken with all video qualities supported. The final reference frame is extracted from each video quality. From the reference frame, the field-of-view (FoV), centering (crop), and aspect ratio are calculated for each video quality.

For the FoV calculation, the test compares the video frame circle size to the calculated circle size from the reference image and video format sizes. For the crop check, the test compares the video frame centering to the reference image centering. For the aspect ratio check, using the scene, which is a large black circle on a white background, the test determines the aspect ratio of the circle and verifies that the frame isn't distorted.

sensor_fusion/test_preview_stabilization.py

Android 13 adds the test_preview_stabilization test, which tests all supported preview qualities up to 1920x1080. The test uses the sensor fusion test rig and requires the new revision 2 sensor fusion controller or a firmware upgrade of earlier Arduino controller versions. For more details about the new controller, see Hardware updates and additions.

Method

The phone is moved slowly over a roughly 15 degree angle with preview stabilization enabled. The rotation of the images is then compared to the rotation of the gyroscope.

sensor_fusion/test_video_stabilization.py

Android 13 adds the test_video_stabilization test, which tests supported video qualities up to 1920x1080. QCIF low resolution video quality is excluded. The test uses the sensor fusion test rig and requires the new revision 2 sensor fusion controller or a firmware upgrade of earlier Arduino controller versions. For more details about the new controller, see Hardware updates and additions.

Method

The test rig moves the phone slowly over a roughly 15 degree angle with preview stabilization enabled. The rotation of the images is then compared to the rotation of the gyroscope.

Hardware updates and additions

Android 13 includes an upgrade to the Arduino-based revision 2 sensor fusion controller. This upgrade includes both hardware and firmware changes. The upgrade allows the controller to set rotation speed and lightning control. The upgrade also includes a new shield (shown in Figure 1), which provides three channels of rotation and three channels of lighting control.

Figure 1. Arduino shield rev2.0 with rotation and lighting control

Additionally, for 13, the firmware of existing controllers must be updated to enable rotation speed control. This firmware can be downloaded from:

• Android 13 Arduino Micro-code VarSpeedServo_and_lighting_control.ino.

For more information on the new controller, see Android 13 sensor fusion controller.

Foldable testing support

Android 13 includes testing support for foldable devices. To enable the multi-camera API for foldables, the phone state (Folded or Unfolded) is required during testing. To enable polling of the phone state for all DUTs, including non-foldable phones, run the following adb command after the DUT is connected to the host.

adb shell am compat enable ALLOW_TEST_API_ACCESS com.android.cts.verifier