Python Ctypes: Invoking C/C++ Shared Library and Native Threading
Python is renowned for its simplicity and ease of use, but when it comes to interacting with C-based shared libraries, Python’s high-level nature may present limitations. To overcome these limitations, developers often need to leverage C extensions or modules to access low-level functionalities. Dynamsoft’s Python Barcode SDK is an example of a CPython extension, specifically built around its C/C++ Barcode SDK, delivering high performance and precision. Python’s Ctypes module offers a different approach that enables direct interaction with low-level C APIs from Python. This article delves into how to use Ctypes to invoke the Dynamsoft C++ Barcode SDK, and further explores how to bypass Python’s Global Interpreter Lock (GIL) by employing native threading.
This article is Part 10 in a 11-Part Series.
- Part 1 - Detecting and Decoding QR Codes in Python with YOLO and Dynamsoft Barcode Reader
- Part 2 - How to a GUI Barcode Reader with Qt PySide6 on Raspberry Pi
- Part 3 - Advanced GUI Python Barcode and QR Code Reader for Windows, Linux, macOS and Rasberry Pi OS
- Part 4 - Advanced QR Code Recognition: Handling Inverted Colors, Perspective Distortion, and Grayscale Images
- Part 5 - Scanning QR Code from Desktop Screen with Qt and Python Barcode SDK
- Part 6 - Building an Online Barcode and QR Code Scanning App with Python Django
- Part 7 - Real-Time Barcode and QR Code Scanning with Webcam, OpenCV, and Python
- Part 8 - How to Build Flet Chat App with Barcode and Gemini APIs
- Part 9 - Comparing Barcode Scanning in Python: ZXing vs. ZBar vs. Dynamsoft Barcode Reader
- Part 10 - Python Ctypes: Invoking C/C++ Shared Library and Native Threading
- Part 11 - A Guide to Running ARM32 and ARM64 Python Barcode Readers in Docker Containers
Prerequisites
- Obtain a valid Dynamsoft Barcode Reader license v9.x for desktop.
- Download the Dynamsoft Barcode Reader C/C++ v9.x packages.
Attempting to Call the C API of Dynamsoft Barcode SDK Directly via Ctypes
In this section, we will demonstrate how to write a simple barcode reading application by directly calling the C API of the Dynamsoft Barcode SDK using Ctypes. The goal is to decode a barcode image and print the barcode format and text to the console. To follow along, you‘ll need to reference DynamsoftBarcodeReader.h
to understand the C API functions.
-
First, load the shared library using
windll
for Windows andCDLL
for Linux:import os import platform from ctypes import * dbr = None if 'Windows' in system: dll_path = license_dll_path = os.path.join(os.path.abspath( '.'), r'..\..\lib\win\DynamsoftBarcodeReaderx64.dll') dbr = windll.LoadLibrary(dll_path) else: dbr = CDLL(os.path.join(os.path.abspath('.'), '../../lib/linux/libDynamsoftBarcodeReader.so'))
-
Next, set the license key and initialize the Dynamsoft Barcode Reader:
license_key = b"LICENSE-KEY" error_msg_buffer = create_string_buffer(256) error_msg_buffer_len = len(error_msg_buffer) ret = dbr.DBR_InitLicense(license_key, error_msg_buffer, error_msg_buffer_len) print('initLicense: {}'.format(ret)) DBR_CreateInstance = dbr.DBR_CreateInstance DBR_CreateInstance.restype = c_void_p instance = dbr.DBR_CreateInstance()
-
Decode a barcode image file:
DBR_DecodeFile = dbr.DBR_DecodeFile DBR_DecodeFile.argtypes = [c_void_p, c_char_p, c_char_p] DBR_DecodeFile.restype = c_int ret = DBR_DecodeFile(instance, c_char_p( 'test.png'.encode('utf-8')), c_char_p(''.encode('utf-8'))) print('DBR_DecodeFile: {}'.format(ret))
-
Define the necessary C structures in Python to handle the decoding results:
class SamplingImageData(Structure): _fields_ = [ ("bytes", POINTER(c_ubyte)), ("width", c_int), ("height", c_int) ] class LocalizationResult(Structure): _fields_ = [ ("terminatePhase", c_int), ("barcodeFormat", c_int), ("barcodeFormatString", c_char_p), ("barcodeFormat_2", c_int), ("barcodeFormatString_2", c_char_p), ("x1", c_int), ("y1", c_int), ("x2", c_int), ("y2", c_int), ("x3", c_int), ("y3", c_int), ("x4", c_int), ("y4", c_int), ("angle", c_int), ("moduleSize", c_int), ("pageNumber", c_int), ("regionName", c_char_p), ("documentName", c_char_p), ("resultCoordinateType", c_int), ("accompanyingTextBytes", c_char_p), ("accompanyingTextBytesLength", c_int), ("confidence", c_int), ("transformationMatrix", c_double * 9), ("reserved", c_char * 52) ] class ExtendedResult(Structure): _fields_ = [ ("resultType", c_int), ("barcodeFormat", c_int), ("barcodeFormatString", c_char_p), ("barcodeFormat_2", c_int), ("barcodeFormatString_2", c_char_p), ("confidence", c_int), ("bytes", POINTER(c_ubyte)), ("bytesLength", c_int), ("accompanyingTextBytes", POINTER(c_ubyte)), ("accompanyingTextBytesLength", c_int), ("deformation", c_int), ("detailedResult", c_void_p), ("samplingImage", SamplingImageData), ("clarity", c_int), ("reserved", c_char * 40) ] class TextResult(Structure): _fields_ = [ ("barcodeFormat", c_int), ("barcodeFormatString", c_char_p), ("barcodeFormat_2", c_int), ("barcodeFormatString_2", c_char_p), ("barcodeText", c_char_p), ("barcodeBytes", POINTER(c_ubyte)), ("barcodeBytesLength", c_int), ("localizationResult", POINTER(LocalizationResult)), ("detailedResult", c_void_p), ("resultsCount", c_int), ("results", POINTER(POINTER(ExtendedResult))), ("exception", c_char_p), ("isDPM", c_int), ("isMirrored", c_int), ("reserved", c_char * 44) ] class TextResultArray(Structure): _fields_ = [ ("resultsCount", c_int), ("results", POINTER(POINTER(TextResult))) ]
-
Finally, retrieve and print the barcode results:
pResults = POINTER(TextResultArray)() DBR_GetAllTextResults = dbr.DBR_GetAllTextResults DBR_GetAllTextResults.argtypes = [c_void_p, POINTER(POINTER(TextResultArray))] DBR_GetAllTextResults.restype = c_int ret = DBR_GetAllTextResults(instance, byref(pResults)) print('DBR_GetAllTextResults: {}'.format(ret)) if ret != 0 or pResults.contents.resultsCount == 0: print("No barcode found.") else: print(f"Total barcode(s) found: {pResults.contents.resultsCount}") for i in range(pResults.contents.resultsCount): result = pResults.contents.results[i] print(result) print(f"Barcode {i+1}:") # crash print(result.contents) print(f" Type: {result.contents.barcodeFormatString.decode('utf-8')}") print(f" Text: {result.contents.barcodeText.decode('utf-8')}")
Unfortunately, the program might crash with a segmentation fault error when trying to print the barcode format and text.
Resolving the Issue with a C Bridging Library
To resolve the issue, we can create a C bridging library to simplify the output data structures and memory management. Additionally, this approach allows us to use native threading to bypass Python’s Global Interpreter Lock (GIL) for improved performance.
Bridging Code for Ctypes and Dynamsoft C/C++ Barcode SDK
We create a CMake library project named bridge
, which contains bridge.cpp
, bridge.h
and CMakeLists.txt
. The CMakeLists.txt
file is as follows:
cmake_minimum_required(VERSION 3.0.0)
project(bridge VERSION 0.1.0)
INCLUDE_DIRECTORIES("${CMAKE_CURRENT_SOURCE_DIR}" "${CMAKE_CURRENT_SOURCE_DIR}/../../../include")
if (CMAKE_HOST_WIN32)
LINK_DIRECTORIES("${CMAKE_CURRENT_SOURCE_DIR}/../../../lib/Win")
else()
LINK_DIRECTORIES("${CMAKE_CURRENT_SOURCE_DIR}/../../../lib/linux")
endif()
add_library(${PROJECT_NAME} SHARED bridge.cpp)
if(CMAKE_HOST_WIN32)
target_link_libraries (${PROJECT_NAME} "DBRx64")
else()
target_link_libraries (${PROJECT_NAME} "DynamsoftBarcodeReader")
endif()
bridge.cpp
is the main source file for the bridging library. It includes the header filebridge.h
and the Dynamsoft Barcode Reader header fileDynamsoftBarcodeReader.h
.- The
bridge.h
file defines the C structures and functions that will be used by Python Ctypes.
The bridge.h
file is as follows:
# include "DynamsoftBarcodeReader.h"
#if !defined(_WIN32) && !defined(_WIN64)
#define EXPORT_API
#else
#define EXPORT_API __declspec(dllexport)
#endif
typedef struct {
char* format;
char* text;
} ResultInfo;
typedef struct {
int size;
ResultInfo** pResultInfo;
} ResultList;
typedef int (*callback_t)(ResultList*);
#ifdef __cplusplus
extern "C" {
#endif
EXPORT_API ResultList* dbr_get_results(void* barcodeReader);
EXPORT_API void dbr_free_results(ResultList* resultList);
EXPORT_API void thread_decode(void* barcodeReader, const char *fileName);
EXPORT_API int registerCallback(callback_t foo);
#ifdef __cplusplus
}
#endif
Explanation
ResultInfo
: A structure that contains the barcode format and text.ResultList
: A structure that contains the size of the result list and a pointer to an array ofResultInfo
structures.dbr_get_results
: A function that retrieves the barcode decoding results, replacingDBR_GetAllTextResults
.dbr_free_results
: A function that frees the memory allocated for the barcode decoding results.thread_decode
: A function that decodes a barcode image file in a separate thread.registerCallback
: A function that registers a Python callback function for receiving the barcode decoding results.
In the bridge.cpp
file, we add the implementation of the functions defined in bridge.h
:
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include "bridge.h"
#include <thread>
using namespace std;
callback_t callback = NULL;
thread t;
ResultList *dbr_get_results(void *barcodeReader)
{
TextResultArray *pResults;
int ret = DBR_GetAllTextResults(barcodeReader, &pResults);
int count = pResults->resultsCount;
TextResult **results = pResults->results;
ResultInfo **pResultInfo = (ResultInfo **)malloc(sizeof(ResultInfo *) * count);
ResultList *resultList = (ResultList *)malloc(sizeof(ResultList));
resultList->size = count;
resultList->pResultInfo = pResultInfo;
for (int i = 0; i < count; i++)
{
TextResult *pResult = results[i];
ResultInfo *pInfo = (ResultInfo *)malloc(sizeof(ResultInfo));
pInfo->format = NULL;
pInfo->text = NULL;
pResultInfo[i] = pInfo;
// printf("Barcode format: %s, text: %s\n", pResult->barcodeFormatString, pResult->barcodeText);
pInfo->format = (char *)calloc(strlen(pResult->barcodeFormatString) + 1, sizeof(char));
strncpy(pInfo->format, pResult->barcodeFormatString, strlen(pResult->barcodeFormatString));
pInfo->text = (char *)calloc(strlen(pResult->barcodeText) + 1, sizeof(char));
strncpy(pInfo->text, pResult->barcodeText, strlen(pResult->barcodeText));
}
DBR_FreeTextResults(&pResults);
return resultList;
}
void dbr_free_results(ResultList *resultList)
{
int count = resultList->size;
ResultInfo **pResultInfo = resultList->pResultInfo;
for (int i = 0; i < count; i++)
{
ResultInfo *resultList = pResultInfo[i];
if (resultList)
{
if (resultList->format != NULL)
free(resultList->format);
if (resultList->text != NULL)
free(resultList->text);
free(resultList);
}
}
if (pResultInfo != NULL)
free(pResultInfo);
}
void thread_func(void *barcodeReader, const char *fileName)
{
DBR_DecodeFile(barcodeReader, fileName, "");
TextResultArray *pResults;
int ret = DBR_GetAllTextResults(barcodeReader, &pResults);
int count = pResults->resultsCount;
TextResult **results = pResults->results;
ResultInfo **pResultInfo = (ResultInfo **)malloc(sizeof(ResultInfo *) * count);
ResultList *resultList = (ResultList *)malloc(sizeof(ResultList));
resultList->size = count;
resultList->pResultInfo = pResultInfo;
for (int i = 0; i < count; i++)
{
TextResult *pResult = results[i];
ResultInfo *pInfo = (ResultInfo *)malloc(sizeof(ResultInfo));
pInfo->format = NULL;
pInfo->text = NULL;
pResultInfo[i] = pInfo;
// printf("Barcode format: %s, text: %s\n", pResult->barcodeFormatString, pResult->barcodeText);
pInfo->format = (char *)calloc(strlen(pResult->barcodeFormatString) + 1, sizeof(char));
strncpy(pInfo->format, pResult->barcodeFormatString, strlen(pResult->barcodeFormatString));
pInfo->text = (char *)calloc(strlen(pResult->barcodeText) + 1, sizeof(char));
strncpy(pInfo->text, pResult->barcodeText, strlen(pResult->barcodeText));
}
DBR_FreeTextResults(&pResults);
if (callback)
{
int res = callback(resultList);
}
}
void thread_decode(void *barcodeReader, const char *fileName)
{
t = thread(thread_func, barcodeReader, fileName);
t.join();
}
int registerCallback(callback_t foo)
{
callback = foo;
return 0;
}
Building the Bridge Library
To build the bridge
library:
On Windows:
cd bridge && mkdir build && cd build
cmake -DCMAKE_GENERATOR_PLATFORM=x64 ..
cmake --build .
On Linux:
cd bridge && mkdir build && cd build
cmake ..
cmake --build .
Loading the Bridge Library in Python
Update the Python code to load the bridge
library:
dbr = None
bridge = None
if 'Windows' in system:
dll_path = license_dll_path = os.path.join(os.path.abspath(
'.'), r'..\..\lib\win\DynamsoftBarcodeReaderx64.dll')
dbr = windll.LoadLibrary(dll_path)
bridge = windll.LoadLibrary(os.path.join(
os.path.abspath('.'), r'bridge\build\Debug\bridge.dll'))
else:
dbr = CDLL(os.path.join(os.path.abspath('.'),
'../../lib/linux/libDynamsoftBarcodeReader.so'))
bridge = CDLL(os.path.join(os.path.abspath(
'.'), 'bridge/build/libbridge.so'))
Note: The library loading sequence is vital for Linux: load libDynamsoftBarcodeReader.so
first, followed by libbridge.so
. Incorrect order may cause the Python code to fail on Linux.
Updated Python Code with Bridge Library
Here is the updated Python code to decode a barcode image file and print the barcode format and text using the bridge
library:
import os
import platform
from ctypes import *
import threading
class ResultInfo(Structure):
_fields_ = [("format", c_char_p), ("text", c_char_p)]
class ResultList(Structure):
_fields_ = [("size", c_int), ("pResultInfo", POINTER(POINTER(ResultInfo)))]
system = platform.system()
dbr = None
bridge = None
if 'Windows' in system:
dll_path = license_dll_path = os.path.join(os.path.abspath(
'.'), r'..\..\lib\win\DynamsoftBarcodeReaderx64.dll')
dbr = windll.LoadLibrary(dll_path)
bridge = windll.LoadLibrary(os.path.join(
os.path.abspath('.'), r'bridge\build\Debug\bridge.dll'))
else:
dbr = CDLL(os.path.join(os.path.abspath('.'),
'../../lib/linux/libDynamsoftBarcodeReader.so'))
bridge = CDLL(os.path.join(os.path.abspath(
'.'), 'bridge/build/libbridge.so'))
# DBR_InitLicense
DBR_InitLicense = dbr.DBR_InitLicense
DBR_InitLicense.argtypes = [c_char_p, c_char_p, c_int]
DBR_InitLicense.restype = c_int
license_key = b"LICENSE-KEY"
error_msg_buffer = create_string_buffer(256)
error_msg_buffer_len = len(error_msg_buffer)
# https://www.dynamsoft.com/customer/license/trialLicense/?product=dcv&package=cross-platform
ret = DBR_InitLicense(license_key, error_msg_buffer, error_msg_buffer_len)
# DBR_CreateInstance
DBR_CreateInstance = dbr.DBR_CreateInstance
DBR_CreateInstance.restype = c_void_p
instance = dbr.DBR_CreateInstance()
######################################################################
# Call decoding method in native thread.
@CFUNCTYPE(None, POINTER(ResultList))
def callback(address):
data = cast(address, POINTER(ResultList))
size = data.contents.size
results = data.contents.pResultInfo
for i in range(size):
result = results[i]
print('Format: %s' % result.contents.format.decode('utf-8'))
print('Text: %s' % result.contents.text.decode('utf-8'))
dbr_free_results = bridge.dbr_free_results
dbr_free_results.argtypes = [c_void_p]
if bool(address):
dbr_free_results(address)
DBR_DestroyInstance = dbr.DBR_DestroyInstance
DBR_DestroyInstance.argtypes = [c_void_p]
DBR_DestroyInstance(instance)
return 0
def run():
print("Python thread" + str(threading.current_thread()))
bridge.registerCallback(callback)
thread_decode = bridge.thread_decode
thread_decode.argtypes = [c_void_p, c_void_p]
thread_decode(instance, c_char_p('test.png'.encode('utf-8')))
t = threading.Thread(target=run)
t.start()
t.join()
This code uses native threading via ctypes to bypass Python’s Global Interpreter Lock (GIL), allowing the barcode decoding to run in a separate thread for better performance. This approach can be particularly useful when dealing with CPU-intensive tasks or I/O-bound operations.
Run the Python code to decode a barcode image file and observe the results printed to the console:
Drawbacks of Using Ctypes
While Ctypes offers flexibility and ease of access to C-based shared libraries, it also comes with several drawbacks:
- Performance Overhead: Ctypes may introduce more overhead compared to C extensions like CPython, particularly for frequently called functions or intensive computations. This is because Ctypes operates through a higher level of abstraction and incurs additional processing costs when converting between Python and C types.
- Manual Memory Management: Ctypes requires the developer to manage memory manually. Incorrect memory handling, such as failing to free allocated memory or incorrectly managing pointers, can lead to memory leaks or crashes.
- Limited Error Handling: Error handling in Ctypes is less robust compared to using a native CPython extension. Errors in Ctypes often manifest as segmentation faults or crashes, which can be harder to debug than the exceptions provided by Python.
- Complex API Usage: Ctypes can be complex and error-prone when dealing with more advanced C features like struct alignment, unions, or callback functions. Developers need to have a solid understanding of both the C API and how to map these to Python using Ctypes.
Conclusion
While ctypes offers a flexible and convenient way to interface with C libraries from Python, it is often better suited for specific scenarios where direct interaction with C functions is necessary. CPython extensions are optimized for performance and provide better integration with Python’s runtime, including more reliable memory management and error handling.
To develop a production-grade Python barcode reader application, it is recommended to use the official CPython extension provided by Dynamsoft.
pip install dbr
Source Code
https://github.com/yushulx/python-barcode-qrcode-sdk/tree/main/examples/ctypes