React Native Image Cropper with Skia and Gesture Handler

Sep 11, 2024

In this article, we are going to write a React Native demo to crop images. It uses React Native Skia and React Native Gesture Handler to provide an interactive cropper.

Dynamsoft Document Normalizer is used to detect the document boundaries and do perspective transformation to crop the image.

cropper

New Project

Create a new React Native project:

npx @react-native-community/cli@latest init ImageCropper

Add Dependencies

Install React Native Skia to draw the polygon representing the boundaries of documents.
```
npm install @shopify/react-native-skia
```
Install React Native Gesture Handler to make it possible to adjust the polygon with gestures.
```
npm install react-native-gesture-handler react-native-reanimated
```
Install React Native Image Picker to pick an image from the album.
```
npm install react-native-image-picker
```

Install vision-camera-dynamsoft-document-normalizer to detect and crop document images.

npm install vision-camera-dynamsoft-document-normalizer react-native-vision-camera react-native-worklets-core

In addition, add the following to babel.conf.js:

 module.exports = {
   presets: ['module:@react-native/babel-preset'],
+  plugins: [
+    'react-native-reanimated/plugin',
+    'react-native-worklets-core/plugin',
+  ],
 };

Pick an Image and Display it in Skia’s Canvas

Use react-native-image-picker to pick an image and we can get its uri and dimensions.

const response = await launchImageLibrary({ mediaType: 'photo'});
const photoUri = response.assets[0].uri;
const photoWidth = response.assets[0].width;
const photoHeight = response.assets[0].height;

Then, draw the image using Skia.

export default function Cropper(props:CropperProps) {
  const image = useImage(props.photo!.photoUri);
  const { width, height } = useWindowDimensions();
  return (
    <Canvas style={{ flex: 1 }}>
      <Fill color="white" />
      <Image image={image} fit="contain" x={0} y={0} width={width} height={height} />
      )}
    </Canvas>
  );
}

Detect the Document and Draw Polygon

Next, detect the document within the selected image and draw its polygon.

Initialize the license of Dynamsoft Document Normalizer. You can apply for a license here.

import * as DDN from 'vision-camera-dynamsoft-document-normalizer';
   
React.useEffect(() => {
  const initLicense = async () => {
    let result = await DDN.initLicense('DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==');
    if (result === false) {
      Alert.alert('','License invalid');
    }else{
      setInitializing(false);
    }
  };
  initLicense();
}, []);

Detect the document and save the detection results as a shared value.

const defaultPoints = [{x:100,y:50},{x:200,y:50},{x:200,y:100},{x:100,y:100}];
const points = useSharedValue(defaultPoints);
let results = await DDN.detectFile(props.photo.photoUri);
let detected = false;
for (let index = 0; index < results.length; index++) {
  const detectedResult = results[index];
  if (detectedResult.confidenceAsDocumentBoundary > 50) {
    points.value = scaledPoints(detectedResult.location.points);
    detected = true;
    break;
  }
}
if (!detected) {
  let photoWidth = props.photo.photoWidth;
  let photoHeight = props.photo.photoHeight;
  let topLeft = {x:photoWidth*0.2,y:photoHeight*0.4};
  let topRight = {x:photoWidth*0.8,y:photoHeight*0.4};
  let bottomRight = {x:photoWidth*0.8,y:photoHeight*0.7};
  let bottomLeft = {x:photoWidth*0.2,y:photoHeight*0.7};
  points.value = scaledPoints([topLeft,topRight,bottomRight,bottomLeft]); //use a preset region
  Alert.alert('','No documents detected');
}

We need to convert the points to match the image displayed on the screen.

const scaledPoints = (detectedPoints:[Point,Point,Point,Point]) => {
  let photoWidth:number = props.photo!.photoWidth;
  let photoHeight:number = props.photo!.photoHeight;
  let newPoints = [];
  let {displayedWidth, displayedHeight} = getDisplayedSize();
  let widthDiff = (width - displayedWidth) / 2;
  let heightDiff = (height - displayedHeight) / 2;
  let xRatio = displayedWidth / photoWidth;
  let yRatio = displayedHeight / photoHeight;
  for (let index = 0; index < detectedPoints.length; index++) {
    const point = detectedPoints[index];
    const x = point.x * xRatio + widthDiff;
    const y = point.y * yRatio + heightDiff;
    newPoints.push({x:x,y:y});
  }
  return newPoints;
};

Draw the polygon using Skia.

const polygonPoints = useDerivedValue(() => {
  return [vec(points.value[0].x,points.value[0].y),
  vec(points.value[1].x,points.value[1].y),
  vec(points.value[2].x,points.value[2].y),
  vec(points.value[3].x,points.value[3].y),
  vec(points.value[0].x,points.value[0].y)];
},[points]);

//...

<Points
  points={polygonPoints}
  mode="polygon"
  color="lightblue"
  style="fill"
  strokeWidth={4}
/>

polygon

Make the Polygon Adjustable using Gesture Handler

Add rectangles around the corners to indicate that the polygon is adjustable.

const [selectedIndex,setSelectedIndex] = useState(-1);
const rectWidth = 10;
const rect1X = useDerivedValue(() => {
  return points.value[0].x - rectWidth;
},[points]);
const rect1Y = useDerivedValue(() => {
  return points.value[0].y - rectWidth;
},[points]);
const rect2X = useDerivedValue(() => {
  return points.value[1].x;
},[points]);
const rect2Y = useDerivedValue(() => {
  return points.value[1].y - rectWidth;
},[points]);
const rect3X = useDerivedValue(() => {
  return points.value[2].x;
},[points]);
const rect3Y = useDerivedValue(() => {
  return points.value[2].y;
},[points]);
const rect4X = useDerivedValue(() => {
  return points.value[3].x - rectWidth;
},[points]);
const rect4Y = useDerivedValue(() => {
  return points.value[3].y;
},[points]);
const rects = () => {
  let rectList = [{x:rect1X,y:rect1Y},{x:rect2X,y:rect2Y},{x:rect3X,y:rect3Y},{x:rect4X,y:rect4Y}];
  const items = rectList.map((rect,index) =>
    <Rect key={'rect-' + index}  style="stroke" strokeWidth={(index === selectedIndex) ? 6 : 4} x={rect.x} y={rect.y} width={rectWidth} height={rectWidth} color="lightblue" />
  );
  return items;
};

polygon with rect

Wrap the component in GestureHandlerRootView.

<GestureHandlerRootView>
  <Cropper photo={photo} onCanceled={()=>setShowCropper(false)} onConfirmed={(path) => displayCroppedImage(path)}/>
</GestureHandlerRootView>

Add GestureDetector for the canvas.

<GestureDetector gesture={composed}>
  <Canvas style={{ flex: 1 }}>
  </Canvas>
</GestureDetector>

The detector detects two gestures: tap and pan. In the tap gesture, we detect which rectangle is tapped. In the pan gesture, move the selected corner.

const panGesture = Gesture.Pan()
  .onChange((e) => {
    let index = selectedIndex;
    if (index !== -1) {
      let newPoints = JSON.parse(JSON.stringify(points.value));
      if (Math.abs(e.changeX) < 5 && Math.abs(e.changeY) < 5) {
        newPoints[index].x = newPoints[index].x + e.changeX;
        newPoints[index].y = newPoints[index].y + e.changeY;
      } 
      points.value = newPoints;
    }
  });

const tapGesture = Gesture.Tap()
  .onBegin((e) => {
    const selectRect = () => {
      let rectList = [{x:rect1X,y:rect1Y},{x:rect2X,y:rect2Y},{x:rect3X,y:rect3Y},{x:rect4X,y:rect4Y}];
      for (let index = 0; index < 4; index++) {
        const rect = rectList[index];
        let diffX = Math.abs(e.absoluteX - rect.x.value);
        let diffY = Math.abs(e.absoluteY - rect.y.value);
        if (diffX < 35 && diffY < 35) {
          runOnJS(setSelectedIndex)(index);
          break;
        }
      }
    };
    selectRect();
  });

const composed = Gesture.Simultaneous(tapGesture, panGesture);

Run Perspective Transformation to Get the Cropped Image

After the adjustment, use Dynamsoft Document Normalizer to run perspective transformation to get the cropped image.

const confirm = async () => {
  if (props.onConfirmed) {
    let location = {points:pointsScaledBack()};
    try {
      let normalizedImageResult = await DDN.normalizeFile(props.photo!.photoUri, location, {saveNormalizationResultAsFile:true});
      if (normalizedImageResult.imageURL) {
        props.onConfirmed(normalizedImageResult.imageURL);
      }
    } catch (error) {
      Alert.alert('','Incorrect Selection');
    }
  }
};

cropped

Source Code

We’ve now completed the demo. Get the source code and have a try: https://github.com/tony-xlh/react-native-image-cropper

LANGUAGES

PLATFORMS

FEATURED