Record a Video using CameraX and Read Barcodes from Video Files

You can read barcodes and QR codes from video files on Android in two ways: decode frames while the video plays, or grab every frame with FFmpeg and decode it offline. This article builds a Kotlin app that records videos with CameraX and then reads barcodes from the recorded files with Dynamsoft Barcode Reader using both approaches.

Reading from video files matters when codes move past the camera (for example along a long shelf) or when you want to evaluate live-scanning performance: a video contains rich information, and a state-of-the-art phone can record 240 FPS footage at 1920x1080.

What you’ll build: an Android app with a CameraActivity that records video with the CameraX VideoCapture use case, and a VideoActivity that decodes the video in two modes: a live-emulation mode that snapshots the playing VideoView with PixelCopy, and a frame mode that extracts every frame with FFmpegFrameGrabber (JavaCV) and decodes each bitmap with CaptureVisionRouter (com.dynamsoft:barcodereaderbundle:11.6.2000).

Key Takeaways

• CameraX video recording needs the VideoCapture use case built on a Recorder with a QualitySelector; recording output goes to a MediaStoreOutputOptions destination.
• Video decoding has two practical modes in the sample: PixelCopy snapshots of the playing VideoView (emulates a live scan) and FFmpegFrameGrabber frame extraction (processes every frame, useful for benchmarks).
• With the current SDK, decode each extracted bitmap through cvr.capture(bitmap, EnumPresetTemplate.PT_READ_BARCODES); the decoded BarcodeResultItem entries provide text and format directly.
• Restrict the barcode formats to what the test needs (EAN-13 and QR Code here) through the simplified settings of the preset template to avoid unnecessary decoding work.

Common Developer Questions

How do I record a video with CameraX?

Build a Recorder with a QualitySelector, create the VideoCapture use case with VideoCapture.withOutput(recorder), bind it to the lifecycle together with the Preview use case, and start recording with videoCapture.output.prepareRecording(...).start(...).

How do I read barcodes from a video file?

Two approaches are implemented in the sample. The first plays the video in a VideoView and copies the current frame with PixelCopy before decoding. The second uses FFmpegFrameGrabber from JavaCV to grab every frame, converts each frame to a rotated bitmap, and decodes it with cvr.capture(bitmap, EnumPresetTemplate.PT_READ_BARCODES).

Why restrict decoding to QR Code and EAN-13?

The sample compares Dynamsoft with ZXing, which is configured with POSSIBLE_FORMATS limited to QR Code and EAN-13. Limiting Dynamsoft to the same formats (EnumBarcodeFormat.BF_EAN_13 or EnumBarcodeFormat.BF_QR_CODE) keeps the comparison fair and speeds up decoding.

Which frame should I decode for a stable result?

In the live-emulation mode, decoding on a timer while the video plays gives the closest match to real live scanning. In the frame mode, decoding every frame and keeping the first successful result mirrors the benchmark statistics shown at the end of this article.

Prerequisites

• Android Studio with the Android SDK (API level 21 or higher).
• An Android device or emulator with a camera to record test videos.
• A Dynamsoft license key - the sample uses a time-limited trial key that requires a network connection on first use.

Get a 30-day free trial license

Record Video using CameraX

Let’s create a new Android project with Android Studio and add video recording function to it. The following parts take some code from Google’s CameraXVideo sample.

Add CameraX Dependencies

Open the project’s build.gradle, add the following to include CameraX:

// CameraX dependencies (first release for video is: "1.1.0-alpha10")
def camerax_version = "1.1.0-beta01"
// The following line is optional, as the core library is included indirectly by camera-camera2
implementation "androidx.camera:camera-core:${camerax_version}"
implementation "androidx.camera:camera-camera2:${camerax_version}"
implementation "androidx.camera:camera-lifecycle:${camerax_version}"
implementation "androidx.camera:camera-video:${camerax_version}"
implementation "androidx.camera:camera-view:${camerax_version}"

Request Permissions

1. Open AndroidManifest.xml, add the following permissions:

    <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.RECORD_AUDIO" />
    <uses-permission
        android:name="android.permission.WRITE_EXTERNAL_STORAGE"
        android:maxSdkVersion="28" />
   

2. In MainActivity, request permissions.

    private var PERMISSIONS_REQUIRED = arrayOf(
            Manifest.permission.CAMERA,
            Manifest.permission.RECORD_AUDIO)
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        // add the storage access permission request for Android 9 and below.
        if (Build.VERSION.SDK_INT <= Build.VERSION_CODES.P) {
            val permissionList = PERMISSIONS_REQUIRED.toMutableList()
            permissionList.add(Manifest.permission.WRITE_EXTERNAL_STORAGE)
            PERMISSIONS_REQUIRED = permissionList.toTypedArray()
        }
   
        if (!hasPermissions(this)) {
            // Request camera-related permissions
            activityResultLauncher.launch(PERMISSIONS_REQUIRED)
        }
    }
   
    private fun hasPermissions(context: Context) = PERMISSIONS_REQUIRED.all {
        ContextCompat.checkSelfPermission(context, it) == PackageManager.PERMISSION_GRANTED
    }
   
    private val activityResultLauncher =
        registerForActivityResult(ActivityResultContracts.RequestMultiplePermissions())
        { permissions ->
            // Handle Permission granted/rejected
            var permissionGranted = true
            permissions.entries.forEach {
                if (it.key in PERMISSIONS_REQUIRED && it.value == false)
                    permissionGranted = false
            }
            if (!permissionGranted) {
                Toast.makeText(this, "Permission request denied", Toast.LENGTH_LONG).show()
            }
        }
   
   

Create a Fullscreen Activity for Video Recording

1. Create a new empty activity named CameraActivity

2. Add CameraX’s preview control in the XML file:

    <androidx.camera.view.PreviewView
        android:id="@+id/previewView"
        android:background="@color/purple_200"
        android:layout_width="0dp"
        android:layout_height="0dp"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintDimensionRatio="V,9:16">
   
    </androidx.camera.view.PreviewView>
   

3. Make the activity full screen:

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_camera)
        val decorView: View = window.decorView
        decorView.systemUiVisibility = (View.SYSTEM_UI_FLAG_LAYOUT_STABLE
                or View.SYSTEM_UI_FLAG_LAYOUT_HIDE_NAVIGATION
                or View.SYSTEM_UI_FLAG_LAYOUT_FULLSCREEN
                or View.SYSTEM_UI_FLAG_HIDE_NAVIGATION
                or View.SYSTEM_UI_FLAG_FULLSCREEN
                or View.SYSTEM_UI_FLAG_IMMERSIVE_STICKY)
   
    }
   

A button in MainActivity is used to launch the CameraActivity.

Start Camera Preview

Let’s display the camera preview in CameraActivity first.

@SuppressLint("UnsafeOptInUsageError")
private suspend fun bindCaptureUsecase() {
    val cameraProvider = ProcessCameraProvider.getInstance(this).await()

    val cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA

    var previewView = findViewById<PreviewView>(R.id.previewView)
    previewView.updateLayoutParams<ConstraintLayout.LayoutParams> {
        val orientation = baseContext.resources.configuration.orientation
        if (orientation == Configuration.ORIENTATION_PORTRAIT) {
            dimensionRatio = "V,9:16"
        }else{
            dimensionRatio = "H,16:9"
        }
    }

    val previewBuilder = Preview.Builder()
    previewBuilder.setTargetAspectRatio(AspectRatio.RATIO_16_9)
    val preview = previewBuilder.build().apply {
        setSurfaceProvider(previewView.surfaceProvider)
    }
    try {
        cameraProvider.unbindAll()
        var camera = cameraProvider.bindToLifecycle(
            this,
            cameraSelector,
            preview
        )
    } catch (exc: Exception) {
        exc.printStackTrace()
        Toast.makeText(context,exc.localizedMessage,Toast.LENGTH_LONG).show()
        goBack()
    }
}

The PreviewView’s layout has to be adjusted according to the orientation.

Record Video with the VideoCapture Use Case

Let’s define a VideoCapture use case to add recording function.

1. Create a Recorder and specifies the output video quality.

    var quality = Quality.HD
    val qualitySelector = QualitySelector.from(quality)
    var recorderBuilder = Recorder.Builder()
    recorderBuilder.setQualitySelector(qualitySelector)
    val recorder = recorderBuilder.build()
   

2. Create a VideoCapture use case using the Recorder.

    videoCapture = VideoCapture.withOutput(recorder)
   

3. Add the VideoCapture use case:

   var camera = cameraProvider.bindToLifecycle(
      this,
      cameraSelector,
   +  videoCapture,
      preview
   )
   

4. Start recording and save the output as a file:

    @SuppressLint("MissingPermission")
    private fun startRecording() {
        // create MediaStoreOutputOptions for our recorder: resulting our recording!
        val name = "CameraX-recording-" +
                SimpleDateFormat(FILENAMEFORMAT, Locale.US)
                    .format(System.currentTimeMillis()) + ".mp4"
        val contentValues = ContentValues().apply {
            put(MediaStore.Video.Media.DISPLAY_NAME, name)
        }
        val mediaStoreOutput = MediaStoreOutputOptions.Builder(
            this.contentResolver,
            MediaStore.Video.Media.EXTERNAL_CONTENT_URI)
            .setContentValues(contentValues)
            .build()
   
        // configure Recorder and Start recording to the mediaStoreOutput.
   
        currentRecording = videoCapture.output
            .prepareRecording(this, mediaStoreOutput)
            .apply { if (audioEnabled) withAudioEnabled() }
            .start(mainThreadExecutor, captureListener)
    }
   

5. If we need to stop recording, just stop it:

    currentRecording!!.stop()
   

Read Barcodes and QR Codes from Video Files

Next, we can record a video containing a QR code and read QR codes from it.

Decode while Playing to Emulate Live Scan

We can use VideoView to play the video and then use PixelCopy to take a snapshot of the current frame for decoding. In this way, we can emulate live scan.

1. Play the video.

    videoView.setVideoURI(uri)
    videoView.start()
   

2. Use PixelCopy to take a snapshot.

    /**
     * Pixel copy to copy SurfaceView/VideoView into BitMap
     * source: https://stackoverflow.com/questions/27434087/how-to-capture-screenshot-or-video-frame-of-videoview-in-android
     */
    @RequiresApi(Build.VERSION_CODES.N)
    fun usePixelCopy(videoView: SurfaceView, callback: (Bitmap?) -> Unit) {
        val bitmap: Bitmap = Bitmap.createBitmap(
            videoView.width,
            videoView.height,
            Bitmap.Config.ARGB_8888
        );
        try {
            // Create a handler thread to offload the processing of the image.
            val handlerThread = HandlerThread("PixelCopier");
            handlerThread.start();
            PixelCopy.request(
                videoView, bitmap,
                PixelCopy.OnPixelCopyFinishedListener { copyResult ->
                    if (copyResult == PixelCopy.SUCCESS) {
                        callback(bitmap)
                    }else{
                        decoding = false
                    }
                    handlerThread.quitSafely();
                },
                Handler(handlerThread.looper)
            )
        } catch (e: IllegalArgumentException) {
            callback(null)
            // PixelCopy may throw IllegalArgumentException, make sure to handle it
            e.printStackTrace()
        }
    }
   

3. Start a timer to take snapshots and decode. A decoding property is used to decide whether to decode based on the status of the previous task.

    @RequiresApi(Build.VERSION_CODES.N)
    private fun decodeVideo(){
        val timer = Timer()
        timer.scheduleAtFixedRate(timerTask {
            try {
                if (videoView.isPlaying) {
                    if (decoding == false) {
                        decoding = true
                        usePixelCopy(videoView){ bitmap: Bitmap? ->
   
                            val bm = rotateBitmaptoFitScreen(bitmap!!)
                            val textResults = decodeBitmap(bm)
                            decoding = false
                               
                        }
                    }
                }
            }catch (exc:Exception) {
                exc.printStackTrace()
            }
        },0,2)
        videoView.start()
    }
   

4. Dynamsoft Barcode Reader is used to decode the bitmap.

    private fun decodeBitmap(bm:Bitmap):ArrayList<String> {
        val results:ArrayList<String> = ArrayList<String>()
        val capturedResult = cvr.capture(bm, EnumPresetTemplate.PT_READ_BARCODES)
        val barcodesResult = capturedResult?.decodedBarcodesResult
        if (barcodesResult != null && barcodesResult.items != null) {
            for (item in barcodesResult.items) {
                results.add(item.text)
            }
        }
        return results
    }
   

PS: How to add Dynamsoft Barcode Reader to the project and initialize a CaptureVisionRouter instance.

1. Add the following to the project’s build.gradle.

    allprojects {
        repositories {
            maven {
                url "https://download2.dynamsoft.com/maven/aar"
            }
        }
    }
   

2. Add the following to the app’s build.gradle.

    implementation 'com.dynamsoft:barcodereaderbundle:11.6.2000'
   

3. In the VideoView’s activity, create an instance of CaptureVisionRouter and set up its settings for video decoding of EAN13 and QR codes.

    private lateinit var cvr: CaptureVisionRouter
   
    private fun initDBR(){
        cvr = CaptureVisionRouter(this)
        try {
            val settings = cvr.getSimplifiedSettings(EnumPresetTemplate.PT_READ_BARCODES)
            val barcodeSettings = settings.barcodeSettings
            if (barcodeSettings != null) {
                barcodeSettings.barcodeFormatIds = EnumBarcodeFormat.BF_EAN_13 or EnumBarcodeFormat.BF_QR_CODE
                barcodeSettings.expectedBarcodesCount = 1
            }
            cvr.updateSettings(EnumPresetTemplate.PT_READ_BARCODES, settings)
        } catch (e: Exception) {
            e.printStackTrace()
        }
    }
   

4. Dynamsoft Barcode Reader requires a license to use. Initialize it in MainActivity with the key you obtained in the prerequisites:

    LicenseManager.initLicense(
        "LICENSE-KEY",
        this
    ) { isSuccessful, e ->
        if (!isSuccessful) {
            e?.printStackTrace()
        }
    }
   

Use FFmpeg to Grab Every Frame and Then Decode

We can also grab every video frame and then decode. Since Android does not provide a built-in API for this, we can use FFmpeg to do this.

1. Install javacv which provides a FFMpeg library for Android.

   Add the following the the app’s build.gradle:

    implementation group: 'org.bytedeco', name: 'javacv', version: "1.5.7"
    implementation group: 'org.bytedeco', name: 'ffmpeg', version: '5.0-1.5.7'
    implementation group: 'org.bytedeco', name: 'ffmpeg', version: '5.0-1.5.7', classifier: 'android-arm64'
    implementation group: 'org.bytedeco', name: 'ffmpeg', version: '5.0-1.5.7', classifier: 'android-x86_64'
   

   Here, we only add the required architectures.

2. Use FFMpegFrameGrabber to grab every frame from video and decode. The video frame may be horizontal, which needs rotation if the video is shot with the phone in portrait mode.

    val inputStream: InputStream? = contentResolver.openInputStream(uri)
    val frameGrabber = FFmpegFrameGrabber(inputStream)
    frameGrabber.start()
    val totalFrames = frameGrabber.lengthInVideoFrames
    //val totalFrames = 2
    val th = thread(start=true) {
        for (i in 0..totalFrames-1) {
            val frame = frameGrabber.grabFrame()
            var bm = AndroidFrameConverter().convert(frame)
            bm = rotateBitmaptoFitScreen(bm)
            decodeBitmap(bm)
        }
        frameGrabber.close()
    }
   

Reading Test

Let’s run a reading test on the following 5-second QR code video.

Result in video mode:

Result in frame mode:

We can see that Dynamsoft Barcode Reader has a good reading rate on this test video.

Common Issues & Edge Cases

• PixelCopy fails on some devices. PixelCopy needs a hardware-accelerated surface; if the copy fails, the sample resets the decoding flag and retries on the next timer tick. On unsupported devices, use the FFmpeg frame mode instead.
• The video frame is rotated. Videos recorded in portrait mode come back rotated. The sample rotates every grabbed or copied bitmap with rotateBitmaptoFitScreen() before decoding, otherwise the barcode orientation can hurt the read rate.
• Decoding every frame is slow. Frame-by-frame decoding with JavaCV converts and processes each frame, so expect lower throughput than live scanning. Use the stop conditions in the sample to bound long benchmark runs.
• No barcodes found in a test video. Keep the code sharp and roughly parallel to the camera plane when recording; blur and extreme perspective reduce the read rate in both decode modes.

Source Code

Get the complete sample project source code on GitHub