Receipt Scanning - New Expensify

New Expensify’s mobile apps feature powerful receipt scanning with automatic data extraction using SmartScan OCR technology.

Overview

Receipt scanning on mobile devices:

Camera Capture: Direct camera access for instant receipt photos
SmartScan OCR: Automatic data extraction from receipt images
Multi-Receipt Support: Scan multiple receipts in succession
Gallery Import: Import existing photos from device
Quality Detection: Automatic blur and quality checking

SmartScan automatically extracts merchant name, date, amount, and currency from receipt images.

Camera Permissions

iOS Camera Access

The app requests camera permissions on first use:

src/components/AttachmentPicker/launchCamera/launchCamera.ios.ts

import {launchCamera as launchCameraImagePicker} from 'react-native-image-picker';
import {PERMISSIONS, request, RESULTS} from 'react-native-permissions';
import type {LaunchCamera} from './types';
import {ErrorLaunchCamera} from './types';

const launchCamera: LaunchCamera = (options, callback) => {
    // Checks current camera permissions and prompts the user
    request(PERMISSIONS.IOS.CAMERA)
        .then((permission) => {
            if (permission !== RESULTS.GRANTED) {
                throw new ErrorLaunchCamera('User did not grant permissions', 'permission');
            }

            launchCameraImagePicker(options, callback);
        })
        .catch((error: ErrorLaunchCamera) => {
            callback({
                errorMessage: error.message,
                errorCode: error.errorCode || 'others',
            });
        });
};

export default launchCamera;

If users deny camera permission, they must enable it in device Settings > New Expensify > Camera.

Android Camera Access

Android uses runtime permission requests:

src/components/AttachmentPicker/launchCamera/launchCamera.android.ts

import {PermissionsAndroid} from 'react-native';
import {launchCamera as launchCameraImagePicker} from 'react-native-image-picker';
import type {LaunchCamera} from './types';
import {ErrorLaunchCamera} from './types';

const launchCamera: LaunchCamera = (options, callback) => {
    // Checks current camera permissions and prompts the user
    PermissionsAndroid.request(PermissionsAndroid.PERMISSIONS.CAMERA)
        .then((permission) => {
            if (permission !== PermissionsAndroid.RESULTS.GRANTED) {
                throw new ErrorLaunchCamera('User did not grant permissions', 'permission');
            }

            launchCameraImagePicker(options, callback);
        })
        .catch((error: ErrorLaunchCamera) => {
            callback({
                errorMessage: error.message,
                errorCode: error.errorCode || 'others',
            });
        });
};

export default launchCamera;

Permission Handling

When permissions are denied:

src/libs/fileDownload/FileUtils.ts

function showCameraPermissionsAlert(translate: LocalizedTranslate) {
    Alert.alert(
        translate('attachmentPicker.cameraPermissionRequired'),
        translate('attachmentPicker.expensifyDoesNotHaveAccessToCamera'),
        [
            {
                text: translate('common.cancel'),
                style: 'cancel',
            },
            {
                text: translate('common.settings'),
                onPress: () => Linking.openSettings(),
            },
        ],
    );
}

On iOS, the app reloads when camera permissions are changed in Settings.

Receipt Capture Flow

Taking a Receipt Photo

import launchCamera from '@components/AttachmentPicker/launchCamera';

function captureReceipt() {
  launchCamera(
    {
      mediaType: 'photo',
      includeBase64: false,
      saveToPhotos: false,
      quality: 0.8,
    },
    (response) => {
      if (response.didCancel) {
        return;
      }
      
      if (response.errorCode) {
        handleError(response.errorMessage);
        return;
      }
      
      // Process receipt image
      const {uri, fileName, type} = response.assets[0];
      uploadReceipt(uri, fileName, type);
    }
  );
}

Receipt Processing

Capture: User takes photo or selects from gallery
Validation: App validates file format and size
Upload: Receipt uploaded to server
SmartScan: Server performs OCR extraction
Review: User reviews and confirms extracted data

SmartScan OCR

Automatic Data Extraction

SmartScan extracts key information:

Merchant Name: Business name from receipt
Date: Transaction date
Amount: Total amount paid
Currency: Detected currency
Category: Suggested expense category
Tax: Tax amount when visible

Receipt Data Structure

interface Receipt {
  receiptID?: number;
  source?: string;          // Receipt image URL
  filename?: string;        // Original filename
  state?: 'SCANREADY' | 'SCANNING' | 'SCANNED' | 'OPEN';
  
  // SmartScan extracted data
  transactionID?: string;
  merchant?: string;
  amount?: number;
  currency?: string;
  created?: string;         // ISO date
  category?: string;
}

Scan States

SCANREADY

Receipt uploaded, queued for processing

SCANNING

SmartScan is actively processing the receipt

SCANNED

Data extraction complete, ready for review

OPEN

User manually entered or edited data

import {isScanRequest, isPendingCardOrScanningTransaction} from '@libs/TransactionUtils';

function TransactionItem({transaction}) {
  const isScanning = isScanRequest(transaction) && 
                     isPendingCardOrScanningTransaction(transaction);
  
  if (isScanning) {
    return <ReceiptScanningIndicator />;
  }
  
  return <TransactionDetails transaction={transaction} />;
}

Handling Scan Results

Receipt Scanning Logic

import {isScanning, isScanRequest} from '@libs/TransactionUtils';
import {getReceiptError} from '@libs/actions/IOU';

function getReceiptError(
    receipt: OnyxEntry<Receipt>,
    filename: string,
    isScanRequest = true,
) {
    // If no receipt or not a scan request, return generic error
    if (isEmptyObject(receipt) || !isScanRequest) {
        return ErrorUtils.getMicroSecondOnyxErrorWithTranslationKey('iou.error.genericCreateFailureMessage');
    }

    // Specific error for receipt scanning issues
    return ErrorUtils.getMicroSecondOnyxErrorObject({
        error: receipt.source?.toString() ?? '',
        source: receipt.source?.toString() ?? '',
        filename,
    });
}

Image Optimization

File Validation

src/libs/ReceiptUtils.ts

import {Str} from 'expensify-common';

function getThumbnailAndImageURIs(transaction, receiptPath, receiptFileName) {
    const filename = transaction?.receipt?.filename ?? receiptFileName ?? '';
    const path = transaction?.receipt?.source ?? receiptPath ?? '';
    
    const isReceiptImage = Str.isImage(filename);
    const isReceiptPDF = Str.isPDF(filename);
    const hasEReceipt = !hasReceiptSource(transaction) && transaction?.hasEReceipt;

    if (hasEReceipt) {
        return {image: ROUTES.ERECEIPT.getRoute(transaction.transactionID)};
    }

    // For local files (just captured), no thumbnail yet
    if (typeof path === 'string' && 
        (path.startsWith('blob:') || path.startsWith('file:'))) {
        return {image: path, isLocalFile: true, filename};
    }

    // Generate thumbnail URLs for uploaded images
    if (isReceiptImage) {
        return {
            thumbnail: `${path}.1024.jpg`,
            thumbnail320: `${path}.320.jpg`,
            image: path,
            filename,
        };
    }

    // PDF thumbnail
    if (isReceiptPDF) {
        return {
            thumbnail: `${path.substring(0, path.length - 4)}.jpg.1024.jpg`,
            thumbnail320: `${path.substring(0, path.length - 4)}.jpg.320.jpg`,
            image: path,
            filename,
        };
    }

    return {isThumbnail: true, image: path, filename};
}

Image Formats

Supported receipt formats:

Images: JPEG, PNG, GIF, HEIC, WebP
Documents: PDF
Max Size: 25 MB per file

HEIC images (iPhone default) are automatically converted to JPEG for compatibility.

Thumbnail Generation

Server automatically generates optimized thumbnails:

320px: List view thumbnails
1024px: Detail view
Original: Full-resolution download

// Receipt URL structure
const receiptUrl = 'https://expensify.com/receipts/w_abcd1234.jpg';
const thumbnail320 = `${receiptUrl}.320.jpg`;
const thumbnail1024 = `${receiptUrl}.1024.jpg`;

Multi-Receipt Scanning

Batch Upload

Scan multiple receipts in quick succession:

function handleMultiReceiptUpload() {
  const receipts = [];
  
  // Capture multiple receipts
  for (let i = 0; i < maxReceipts; i++) {
    await captureReceipt().then((receipt) => {
      receipts.push(receipt);
    });
  }
  
  // Upload all receipts
  receipts.forEach((receipt) => {
    createTransaction(receipt);
  });
}

Draft Management

import {removeDraftTransaction} from '@libs/actions/TransactionEdit';

// Remove draft transactions created during multi-scanning
function cleanupDraftTransactions(transactionIDs: string[]) {
  transactionIDs.forEach((id) => {
    removeDraftTransaction(id);
  });
}

Receipt Sources

Camera Capture

const receipt = {
  source: 'file:///path/to/receipt.jpg',  // Local file
  filename: 'receipt.jpg',
  type: 'image/jpeg',
  isLocalFile: true,
};

Gallery Import

import {launchImageLibrary} from 'react-native-image-picker';

function importFromGallery() {
  launchImageLibrary(
    {
      mediaType: 'photo',
      includeBase64: false,
      quality: 0.8,
    },
    (response) => {
      if (!response.didCancel && response.assets) {
        uploadReceipt(response.assets[0]);
      }
    }
  );
}

Remote Receipt

After upload, receipt has remote source:

const receipt = {
  source: 'https://expensify.com/receipts/w_abcd1234.jpg',
  receiptID: 12345,
  filename: 'w_abcd1234.jpg',
  state: 'SCANNING',
};

Fallback Construction

When source is missing but filename exists:

src/libs/ReceiptUtils.ts

function constructReceiptSourceFromFilename(filename: string): string {
    return `${CONFIG.EXPENSIFY.RECEIPTS_URL}${filename}`;
}

// Usage
const fallbackSource = !transaction.receipt?.source && receiptFilename
    ? constructReceiptSourceFromFilename(receiptFilename)
    : undefined;
    
const path = transaction.receipt?.source ?? fallbackSource ?? '';

Error Handling

Common Issues

Camera permission denied

Show alert guiding user to Settings:

if (error.errorCode === 'permission') {
  showCameraPermissionsAlert(translate);
}

Image too large

if (fileSize > MAX_FILE_SIZE) {
  Alert.alert(
    translate('attachmentPicker.fileTooLarge'),
    translate('attachmentPicker.maxSize', {size: '25MB'})
  );
}

Unsupported format

const supportedFormats = ['jpg', 'jpeg', 'png', 'pdf', 'heic'];
if (!supportedFormats.includes(fileExtension)) {
  Alert.alert(translate('attachmentPicker.unsupportedFormat'));
}

SmartScan failed

if (receipt.state === 'SCANFAILED') {
  // Allow manual entry
  setManualEntry(true);
}

Retry Logic

function retryReceiptUpload(transactionID: string, receipt: Receipt) {
  const maxRetries = 3;
  let attempts = 0;
  
  const upload = async () => {
    try {
      await API.write('UploadReceipt', {transactionID, receipt});
    } catch (error) {
      attempts++;
      if (attempts < maxRetries) {
        setTimeout(upload, Math.pow(2, attempts) * 1000);
      } else {
        handleUploadFailure(error);
      }
    }
  };
  
  upload();
}

Offline Behavior

Receipt scanning works offline:

Capture: Receipt photo saved locally
Queue: Upload queued for when online
Display: Local image shown immediately
Sync: Automatic upload when connection restored
SmartScan: Processes once uploaded

import {isOffline} from '@libs/Network/NetworkStore';

function uploadReceipt(receipt: Receipt) {
  if (isOffline()) {
    // Queue for later
    queueReceiptUpload(receipt);
    showOfflineNotice();
  } else {
    // Upload immediately
    uploadReceiptToServer(receipt);
  }
}

See Offline Mode for details on offline architecture.

Testing Receipt Scanning

Test Receipts

Use test receipts for development:

import ReceiptGeneric from '@assets/images/receipt-generic.png';

if (__DEV__) {
  const testReceipt = {
    source: ReceiptGeneric,
    filename: 'test-receipt.png',
    isTestReceipt: true,
  };
}

Mock SmartScan

if (__DEV__ && Config.MOCK_SMARTSCAN) {
  const mockScannedData = {
    merchant: 'Test Merchant',
    amount: 42.00,
    currency: 'USD',
    created: '2024-01-15',
    category: 'Meals & Entertainment',
  };
}

Best Practices

Good Lighting

Ensure receipts are well-lit for better OCR accuracy

Full Receipt Visible

Capture entire receipt including merchant name and total

Avoid Shadows

Position camera to minimize shadows and glare

Flat Surface

Lay receipt flat for optimal scanning

Review Extracted Data

Always verify SmartScan results before submitting

Next Steps

iOS Setup

Configure iOS camera permissions

Android Setup

Configure Android camera permissions

Offline Mode

Offline receipt handling

Create Expenses

Learn about expense management

Platforms

Mobile Features

​Overview

​Camera Permissions

​iOS Camera Access

​Android Camera Access

​Permission Handling

​Receipt Capture Flow

​Taking a Receipt Photo

​Receipt Processing

​SmartScan OCR

​Automatic Data Extraction

​Scan States

​Handling Scan Results

​Image Optimization

​File Validation

​Image Formats

​Thumbnail Generation

​Multi-Receipt Scanning

​Batch Upload

​Draft Management

​Receipt Sources

​Camera Capture

​Gallery Import

​Remote Receipt

​Fallback Construction

​Error Handling

​Common Issues

​Retry Logic

​Offline Behavior

​Testing Receipt Scanning

​Test Receipts

​Mock SmartScan

​Best Practices

​Next Steps

iOS Setup

Android Setup

Offline Mode

Create Expenses

Build docs developers (and LLMs) love

Overview

Camera Permissions

iOS Camera Access

Android Camera Access

Permission Handling

Receipt Capture Flow

Taking a Receipt Photo

Receipt Processing

SmartScan OCR

Automatic Data Extraction

Scan States

Handling Scan Results

Image Optimization

File Validation

Image Formats

Thumbnail Generation

Multi-Receipt Scanning

Batch Upload

Draft Management

Receipt Sources

Camera Capture

Gallery Import

Remote Receipt

Fallback Construction

Error Handling

Common Issues

Retry Logic

Offline Behavior

Testing Receipt Scanning

Test Receipts

Mock SmartScan

Best Practices

Next Steps