How to Build a React Native ID Card Scanner

Feb 28, 2024

Identity documents are adopted in many countries to prove a person’s identity. They usually have a card design around 8.6 cm x 5.4 cm in size with three-line MRZ, like ID cards in Germany and the Netherlands.

Dutch ID Card Specimen:

Dutch ID Card

In this article, we are going to talk about how to build an ID card scanner in React Native. The scanner can capture the front and the back of the ID card and extract the card owner’s info by recognizing the MRZ using OCR. Dynamsoft Label Recognizer is used to provide the OCR functionality.

Demo video:

New Project

Create a new React Native project:

npx react-native@latest init IDCardScanner

Add Dependencies

Install react-native-vision-camera for accessing the camera.
```
npm install react-native-vision-camera
```
Install vision-camera-dynamsoft-label-recognizer for recognizing the MRZ.
```
npm install vision-camera-dynamsoft-label-recognizer
```

Install vision-camera-cropper to crop camera frames.

npm install vision-camera-cropper react-native-worklets-core

Install react-native-svg for drawing the rectangle for cropping.
```
npm install react-native-svg
```

Install react-navigation for navigation.

npm install @react-navigation/native @react-navigation/native-stack react-native-safe-area-context react-native-screens

Install @react-native-async-storage/async-storage to store the scanned ID cards.
```
npm install @react-native-async-storage/async-storage
```
Install mrz for parsing MRZ.
```
npm install mrz
```

Add Camera Permission

For iOS, add the following to Info.plist.

<key>NSCameraUsageDescription</key>
<string>For document scanning</string>

For Android, add the following to AndroidManifest.xml.

<uses-permission android:name="android.permission.CAMERA" />

The app will have three screens.

Home Screen. We can manage scanned ID cards on this page.
Card Screen. We can scan a new ID card or view and modify an ID card on this page.
Camera Screen. We can open the camera to crop the ID card image on this page.

We use react navigation to manage the screens and navigation.

Create screen files under src/screens: HomeScreen.tsx, CardScreen.tsx,CameraScreen.tsx.

Update App.tsx to use react navigation.

import React from 'react';
import { NavigationContainer } from '@react-navigation/native';
import { createNativeStackNavigator } from '@react-navigation/native-stack';
import HomeScreen from './screens/HomeScreen';
import CameraScreen from './screens/CameraScreen';
import CardScreen from './screens/CardScreen';
import { TextButton } from './components/TextButton';

const Stack = createNativeStackNavigator();

function App(): React.JSX.Element {
  return (
    <NavigationContainer>
      <Stack.Navigator>
        <Stack.Screen name="Home" component={HomeScreen} />
        <Stack.Screen name="Camera" component={CameraScreen} />
        <Stack.Screen name="Card" component={CardScreen} />
      </Stack.Navigator>
    </NavigationContainer>
  );
}

export default App;

How the ID Cards are Stored

A ScannedCard interface is defined to represent an ID card. It contains the base64 of its back and front images and the extracted info.

export interface ParsedResult {
  Surname:string,
  GivenName:string,
  IDNumber:string,
  DateOfBirth:string,
  DateOfExpiry:string
}

export interface ScannedIDCard {
  backImage:string,
  frontImage:string,
  info:ParsedResult,
  timestamp:number
}

An IDCardManager is used to store the ID cards using async-storage. The timestamp is used as the key.

import AsyncStorage from '@react-native-async-storage/async-storage';
export class IDCardManager {

  static async saveIDCard(card:ScannedIDCard) {
    await AsyncStorage.setItem(card.timestamp.toString(),JSON.stringify(card));
  }

  static async deleteIDCard(key:string) {
    await AsyncStorage.removeItem(key);
  }

  static async getKeys(){
    return await AsyncStorage.getAllKeys();
  }

  static async getIDCard(key:string){
    let jsonStr:string|null = await AsyncStorage.getItem(key);
    if (jsonStr) {
      let card:ScannedIDCard = JSON.parse(jsonStr);
      return card;
    }else{
      return null;
    }
  }

  static async listIDCards(){
    let cards:ScannedIDCard[] = [];
    let keys = await this.getKeys();
    for (let index = 0; index < keys.length; index++) {
      const key = keys[index];
      let card = await this.getIDCard(key);
      if (card) {
        cards.push(card);
      }
    }
    return cards;
  }
}

Home Screen

Define a Card component under src/components/Card.tsx. It can be listed on the home screen.

export interface CardProps{
  cardKey:string;
  onPress?:()=>void;
}
export function Card(props:CardProps){
  const [card,setCard] = useState<ScannedIDCard|null>();
  const [pressed,setPressed] = useState(false);
  useEffect(() => {
    (async () => {
      console.log("mounted")
      const result = await IDCardManager.getIDCard(props.cardKey);
      setCard(result);
    })();
  }, []);

  const getDate = () => {
    if (card) {
      let timestamp = card.timestamp;
      let date = new Date(timestamp);
      return date.toUTCString();
    }
    return "";
  }

  const getCardDetailsText = () => {
    let text = "Name: "+card?.info.GivenName+" "+card?.info.Surname + "\n";
    text = text + "Scanned Date: " + getDate();
    return text;
  }

  return (
    <Pressable 
      onPress={props.onPress}
      onPressIn={()=>setPressed(true)}
      onPressOut={()=>setPressed(false)}
    >
      <View style={[styles.card,pressed?styles.pressed:null]}>
        <Image 
          style={styles.cardImage}
          source={{
            uri: 'data:image/jpeg;base64,'+card?.frontImage,
          }}
        />
        <View style={styles.cardDetails}>
          <Text>{getCardDetailsText()}</Text>
        </View>
      </View>
    </Pressable>
  )
}

const styles = StyleSheet.create({
  card:{
    flex:1,
    display:"flex",
    flexDirection:"row",
    margin: 10,
    padding:10,
    borderColor:"gray",
    borderWidth:0.2,
    borderRadius:3
  },
  pressed:{
    backgroundColor:"lightgray",
  },
  cardImage:{
    width: 100,
    height: 70,
    resizeMode:"cover"
  },
  cardDetails:{
    flex:1,
    padding:10,
    justifyContent:"center",
    flexDirection:"row"
  },
});

List the existing cards on the home screen.

interface HomeScreenProps {
  route:any;
  navigation:any;
}

export default function HomeScreen(props:HomeScreenProps){
  const selectedCardKey = useRef("");
  const [cardKeys,setCardKeys] = useState<readonly string[]>([]);
  useEffect(() => {
    const unsubscribe = props.navigation.addListener('focus', async () => {
      console.log("screen focused");
      setCardKeys([]);//force rendering
      setCardKeys(await IDCardManager.getKeys());
    });
    return unsubscribe;
  }, [props.navigation]);

  const cardPressed = (key:string) => {
    selectedCardKey.current = key;
  }

  const renderCards = () => {
    let cards:React.ReactElement[] = [];
    if (cardKeys.length == 0) {
      return;
    }
    cardKeys.forEach(async cardKey =>  {
      let card = <Card key={cardKey} cardKey={cardKey} onPress={()=>cardPressed(cardKey)}></Card>;
      cards.push(card);
    });
    if (cards.length>0) {
      return cards;
    }
  }
     
  return (
    <View style={StyleSheet.absoluteFill}>
      <ScrollView style={styles.cardList}>
        {renderCards()}
      </ScrollView>
    </View>
  )
}

When the card is pressed, display a modal for the user to choose whether to open the card or delete the card.

JSX:

<Modal
  animationType="slide"
  transparent={true}
  visible={modalVisible}
  onRequestClose={() => {
    setModalVisible(!modalVisible);
  }}>
  <View style={styles.centeredView}>
    <View style={styles.modalView}>
      <Text style={styles.modalText}>Please select an action:</Text>
      <View style={{flexDirection:"row"}}>
        <Pressable
          style={styles.button}
          onPress={() => performAction("open")}>
          <Text style={styles.textStyle}>Open</Text>
        </Pressable>
        <Pressable
          style={styles.button}
          onPress={() => performAction("delete")}>
          <Text style={styles.textStyle}>Delete</Text>
        </Pressable>
      </View>
    </View>
  </View>
</Modal>

Functions:

const [modalVisible,setModalVisible] = useState(false);
const goToCardScreen = () => {
  console.log("goToCardScreen");
  props.navigation.navigate('Card',{
    cardKey: selectedCardKey.current,
  });
}

const cardPressed = (key:string) => {
  selectedCardKey.current = key;
  setModalVisible(true);
}
   
const performAction = async (mode:"delete"|"open") => {
  if (mode === "delete") {
    await IDCardManager.deleteIDCard(selectedCardKey.current);
    setCardKeys(await IDCardManager.getKeys());
  }else{
    goToCardScreen();
  }
  setModalVisible(!modalVisible);
}

Add a bottom bar with a “scan” button which navigates to the card screen.

JSX:

<View style={[styles.bottomBar, styles.elevation,styles.shadowProp]}>
  <Pressable onPress={()=>{selectedCardKey.current ="";goToCardScreen()}}>
    <View style={styles.circle}>
      <Text style={styles.buttonText}>SCAN</Text>
    </View>
  </Pressable>
</View>

Styles:

const styles = StyleSheet.create({
  bottomBar:{
    width: "100%",
    height: 45,
    marginTop: 5,
    flexDirection:"row",
    justifyContent:"center",
    backgroundColor:"white",
  },
  shadowProp: {
    shadowColor: '#171717',
    shadowOffset: {width: 2, height: 4},
    shadowOpacity: 0.2,
    shadowRadius: 3,
  },
  elevation: {
    elevation: 20,
    shadowColor: '#52006A',
  },
  circle: {
    width: 60,
    height: 60,
    borderRadius: 60 / 2,
    backgroundColor: "rgb(120,190,250)",
    top:-25,
    justifyContent:"center",
  },
  buttonText:{
    alignSelf:"center",
    color:"white",
  },
});

Camera Screen

On camera screen, use Vision Camera to open the camera, add a rectangle to indicate which region to crop and a button for capturing.

const [hasPermission, setHasPermission] = useState(false);
const [isActive,setIsActive] = useState(true);
const device = useCameraDevice("back");
const format = useCameraFormat(device, [
  { videoResolution: { width: 1920, height: 1080 } },
  { fps: 30 }
])
useEffect(() => {
  (async () => {
    const status = await Camera.requestCameraPermission();
    setHasPermission(status === 'granted');
    setIsActive(true);
  })();
}, []);
  
return (
  <View style={StyleSheet.absoluteFill}>
    {device != null &&
    hasPermission && (
    <>
      <Camera
        style={StyleSheet.absoluteFill}
        isActive={isActive}
        device={device}
        format={format}
        frameProcessor={frameProcessor}
        pixelFormat='yuv'
      />
       <Svg preserveAspectRatio={(Platform.OS == 'ios') ? '':'xMidYMid slice'} style={StyleSheet.absoluteFill} viewBox={getViewBox()}>
        <Rect 
          x={cropRegion.left/100*getFrameSize().width}
          y={cropRegion.top/100*getFrameSize().height}
          width={cropRegion.width/100*getFrameSize().width}
          height={cropRegion.height/100*getFrameSize().height}
          strokeWidth="2"
          stroke="red"
          fillOpacity={0.0}
        />
      </Svg>
    </>
    )}
    <View style={[styles.bottomBar]}>
      <Pressable 
        onPressIn={()=>{setPressed(true)}}
        onPressOut={()=>{setPressed(false)}}
        onPress={()=>{capture()}}>
        <View style={styles.outerCircle}>
        <View style={[styles.innerCircle, pressed ? styles.circlePressed:null]}></View>
        </View>
      </Pressable>
    </View>
  </View>
)

The crop region is set based on the ratio of an ID card.

const [cropRegion,setCropRegion] = useState({
  left: 10,
  top: 20,
  width: 80,
  height: 30
});
const cropRegionShared = useSharedValue<undefined|CropRegion>(undefined);

const adaptCropRegionForIDCard = () => {
  let size = getFrameSize();
  let regionWidth = 0.8*size.width;
  let desiredRegionHeight = regionWidth/(85.6/54);
  let height = Math.ceil(desiredRegionHeight/size.height*100);
  let region = {
    left:10,
    width:80,
    top:20,
    height:height
  };
  setCropRegion(region);
  cropRegionShared.value = region;
}
   
const getFrameSize = ():{width:number,height:number} => {
  let size = {width:1080,height:1920};
  return size;
}
     
useEffect(() => {
  (async () => {
    adaptCropRegionForIDCard();
  })();
}, []);

Define a frame processor to capture a frame when the capture button is pressed. It will return to the previous screen with the cropped frame’s base64.

const shouldTake = useSharedValue(false);
const [pressed,setPressed] = useState(false);
const capture = () => {
  shouldTake.value=true;
}
   
const onCaptured = (base64:string) => {
  setIsActive(false);
  if (props) {
    if (props.navigation) {
      props.navigation.navigate({
        name: 'Card',
        params: { base64: base64 },
        merge: true,
      });
    }
  }
}

const onCapturedJS = Worklets.createRunInJsFn(onCaptured);
const frameProcessor = useFrameProcessor((frame) => {
  'worklet';
  if (shouldTake.value == true && cropRegionShared.value != undefined) {
    shouldTake.value = false;
    const result = crop(frame,{cropRegion:cropRegion,includeImageBase64:true,saveAsFile:false});
    if (result.base64) {
      onCapturedJS(result.base64);
    }
  }
}, []);

Card Screen

On the card screen, display the card’s images and info and allow editing of the card.

Define the JSX:

const isFrontRef = useRef(false);
const goToCameraScreen = (isFront:boolean) => {
  isFrontRef.current = isFront;
  props.navigation.navigate('Camera');
}
const Card = (props:{isFront:boolean}) => {
  let base64;
  if (props.isFront) {
    base64 = frontImageBase64;
  }else{
    base64 = backImageBase64;
  }
  let innerControl;
  if (!base64) {
    innerControl = 
      <View style={styles.buttonContainer}>
        <Button title="Add Image"
          onPress={()=>{goToCameraScreen(props.isFront)}}
        ></Button>
      </View>
  }else{
    innerControl = 
      <View style={styles.imageContainer}>
        <Pressable
          onPress={()=>{goToCameraScreen(props.isFront)}}
        >
          <Image 
            style={styles.cardImage}
            source={{
            uri: 'data:image/jpeg;base64,'+base64,
          }}></Image>
        </Pressable>
      </View>
  }
  return (
    <>
      <Text style={styles.header}>
        {props.isFront?"Front":"Back"} Image:
      </Text>
      {innerControl}
    </>
  )
}

const Fields = () => {
  const onChangeText = (key:string,text:string) => {
    console.log("onChangeText");
    let result:any = JSON.parse(JSON.stringify(parsedResult));
    result[key] = text;
    setParsedResult(result);
  }
  let fieldArray = [];
  let keys = Object.keys(parsedResult);
  for (let index = 0; index < keys.length; index++) {
    let key = keys[index];
    const value = (parsedResult as any)[key];
    let view = 
    <View style={styles.infoField} key={"field-"+key}>
      <Text style={styles.fieldLabel}>{key+":"}</Text>
      <TextInput 
        style={styles.fieldInput} 
        onChangeText={(text)=>{onChangeText(key,text)}}
        value={value}/>
    </View>
    fieldArray.push(view);
  }
  return (
    fieldArray
  )
}

return (
  <View style={StyleSheet.absoluteFill}>
    <ScrollView>
      {Card({isFront:true})}
      {Card({isFront:false})}
      <Text style={styles.header}>
        Info
      </Text>
      {Fields()}
    </ScrollView>
  </View>
)

When the component mounted, load the card info if a key is specified.

const cardKey = useRef("");
const [frontImageBase64,setFrontImageBase64] = useState("");
const [backImageBase64,setBackImageBase64] = useState("");
const [parsedResult,setParsedResult] = useState<ParsedResult>(
  {
    Surname:"",
    GivenName:"",
    IDNumber:"",
    DateOfBirth:"",
    DateOfExpiry:""
  }
);
useEffect(() => {
  const init = async () => {
    console.log(props);
    let key = props.route.params.cardKey;
    if (key) {
      cardKey.current = key;
      let IDCard = await IDCardManager.getIDCard(key);
      if (IDCard) {
        setFrontImageBase64(IDCard.frontImage);
        setBackImageBase64(IDCard.backImage);
        setParsedResult(IDCard.info);
      }
    }
  }
  init()
}, []);

When the back image is captured, try to recognize the MRZ lines of the ID card and extract the info.

useEffect(() => {
  if (props.route.params?.base64) {
    let base64 = props.route.params?.base64; 
    if (isFrontRef.current === true) {
      setFrontImageBase64(base64);
    }else{
      setBackImageBase64(base64);
      recognizeIDCard(base64);
    }
  }
}, [props.route.params?.base64]);
   
const recognizeIDCard = async (base64:string) => {
  const result = await decodeBase64(base64)
  if (result.results.length>0) {
    let lineResults = result.results[0].lineResults;
    if (lineResults.length >= 3) {
      let MRZLines = [];
      MRZLines.push(lineResults[lineResults.length - 3].text);
      MRZLines.push(lineResults[lineResults.length - 2].text);
      MRZLines.push(lineResults[lineResults.length - 1].text);
      console.log(MRZLines);
      let parsed = parse(MRZLines);
      let result = {
        Surname:parsed.fields.lastName ?? "",
        GivenName:parsed.fields.firstName ?? "",
        IDNumber:parsed.fields.documentNumber ?? "",
        DateOfBirth:parsed.fields.birthDate ?? "",
        DateOfExpiry:parsed.fields.expirationDate ?? ""
      }
      setParsedResult(result);
      return;
    }
  }
  Alert.alert("","Failed to recognize the card.");
}

Save the scanned card when the “save” button on the header is pressed.

useEffect(() => {
  // Use `setOptions` to update the button that we previously specified
  // Now the button includes an `onPress` handler to update the count
  props.navigation.setOptions({
    headerRight: () => (
      <TextButton title="Save" onPress={()=>saveCard()}></TextButton>
    ),
  });
}, [props.navigation,parsedResult,frontImageBase64,backImageBase64]);

const saveCard = async () => {
  let complete = isInfoComplete();
  if (complete) {
    let key;
    if (cardKey.current) {
      key = parseInt(cardKey.current);
    }else{
      key = new Date().getTime();
      cardKey.current = key.toString();
    }
    let card:ScannedIDCard = {
      frontImage:frontImageBase64,
      backImage:backImageBase64,
      info:parsedResult,
      timestamp:key
    }
    await IDCardManager.saveIDCard(card);
    Alert.alert("","Saved");
  }else{
    Alert.alert("","Card info not complete");
  }
}

Here are the steps to configure Dynamsoft Label Recognize for recognizing MRZ.

Put MRZ model files in the project. You can find the files here.

For Android, we have to put the MRZ model files under assets.

For iOS, add the MRZ model folder as a reference.

Initialize Dynamsoft Label Recognizer with a license and update its template. You can apply for a license here.

useEffect(() => {
  (async () => {
    let success = await initLicense("DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==");
    if (!success) {
      Alert.alert("","License for the MRZ Reader is invalid.");
    }
    try {
      await useCustomModel({customModelFolder:"MRZ",customModelFileNames:["MRZ"]});
      await updateTemplate("{\"CharacterModelArray\":[{\"DirectoryPath\":\"\",\"Name\":\"MRZ\"}],\"LabelRecognizerParameterArray\":[{\"Name\":\"default\",\"ReferenceRegionNameArray\":[\"defaultReferenceRegion\"],\"CharacterModelName\":\"MRZ\",\"LetterHeightRange\":[5,1000,1],\"LineStringLengthRange\":[30,44],\"LineStringRegExPattern\":\"([ACI][A-Z<][A-Z<]{3}[A-Z0-9<]{9}[0-9][A-Z0-9<]{15}){(30)}|([0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z<]{3}[A-Z0-9<]{11}[0-9]){(30)}|([A-Z<]{0,26}[A-Z]{1,3}[(<<)][A-Z]{1,3}[A-Z<]{0,26}<{0,26}){(30)}|([ACIV][A-Z<][A-Z<]{3}([A-Z<]{0,27}[A-Z]{1,3}[(<<)][A-Z]{1,3}[A-Z<]{0,27}){(31)}){(36)}|([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{8}){(36)}|([PV][A-Z<][A-Z<]{3}([A-Z<]{0,35}[A-Z]{1,3}[(<<)][A-Z]{1,3}[A-Z<]{0,35}<{0,35}){(39)}){(44)}|([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{14}[A-Z0-9<]{2}){(44)}\",\"MaxLineCharacterSpacing\":130,\"TextureDetectionModes\":[{\"Mode\":\"TDM_GENERAL_WIDTH_CONCENTRATION\",\"Sensitivity\":8}],\"Timeout\":9999}],\"LineSpecificationArray\":[{\"BinarizationModes\":[{\"BlockSizeX\":30,\"BlockSizeY\":30,\"Mode\":\"BM_LOCAL_BLOCK\",\"MorphOperation\":\"Close\"}],\"LineNumber\":\"\",\"Name\":\"defaultTextArea->L0\"}],\"ReferenceRegionArray\":[{\"Localization\":{\"FirstPoint\":[0,0],\"SecondPoint\":[100,0],\"ThirdPoint\":[100,100],\"FourthPoint\":[0,100],\"MeasuredByPercentage\":1,\"SourceType\":\"LST_MANUAL_SPECIFICATION\"},\"Name\":\"defaultReferenceRegion\",\"TextAreaNameArray\":[\"defaultTextArea\"]}],\"TextAreaArray\":[{\"Name\":\"defaultTextArea\",\"LineSpecificationNameArray\":[\"defaultTextArea->L0\"]}]}");
    } catch (error:any) {
      console.log(error);
      Alert.alert("Error","Failed to load model.");
    }
  })();
}, []);

Source Code

All right, we have covered the key parts of the React Native ID card scanner demo. Check out the source code to have a try:

https://github.com/tony-xlh/react-native-id-card-scanner

LANGUAGES

PLATFORMS

FEATURED

How to Build a React Native ID Card Scanner

New Project

Add Dependencies

Add Camera Permission

Navigation

How the ID Cards are Stored

Home Screen

Camera Screen

Card Screen

Source Code

Search Blog Posts

POPULAR TOPICS

LANGUAGES

PLATFORMS

FEATURED

How to Build a React Native ID Card Scanner

New Project

Add Dependencies

Add Camera Permission

Navigation

How the ID Cards are Stored

Home Screen

Camera Screen

Card Screen

Source Code

Search Blog Posts

POPULAR TOPICS

Useful Links