Speedy Document Scanning Directly into FileMaker Pro 13

Fujitsu arguably makes the best portable document scanners. I have one at home and love it. The sad fact is that Fujitsu, though they make many professional document scanners, do not support TWAIN, the industry standard API for software integration.

I went looking for FileMaker integration solutions and didn’t see anything that fit the bill (more on this below in the Other Approaches section).

Setting a stop watch on the process of scanning, here is what I found. With the scanner set to single-sided pages and 150 dpi gray scale images, (average 115 kb sized files) it took:

• 46 seconds –  to scan 10 pages through the scanner
• 52 seconds –  have all the PDFs imported into a FileMaker container field.

As you can see, most of the time of the process is spent scanning. This may be faster or slower depending on your chosen document size, scanner speed, scanner settings and how fast your computer is running. The Automator Folder Action responds almost immediately to new items being scanned or dropped into it.

Download Demo File

1. Create a new profile, give it a meaningful name, such as ‘Scan to ScanSnap Folder’
2. Set the Application to run as ‘none’.
3. Turn off Quick Menu

Click the Browse button to choose a folder that you want your scans to go to.

You can also change the File Name Format on this screen.

Because the workflow involves a single page document, I opted for Simplex scanning (single sided) and a fairly low resolution (150 dpi) to keep the file size down.

In the next tab – File Option – I selected ‘.PDF’ for the File Format. There is an Option dialog box here that allows  you to set ‘Generate one PDF file per page’. If you are scanning double-sided, you can change the default to 2 pages/PDF.

From the Dock select ‘Scan to ScanSnap Folder’, or whatever you named your ScanSnap Profile. To get back to the Settings for Scansnap Manager, hold down the Control Key when selecting the ScanSnap icon from the Dock.

• Click on the ‘Info’ icon (the small ‘i’ in the top right corner of the main screen in the demo file.
• Save the Workflow file from the Info box in the demo file.
• Open it in Automator.
• Copy the entire script and paste it into a new Folder Action script.
• Set the Target Folder

Click the Open with Automator button.

Here is what the Automator routine does:

1. Set the Folder Action to attach it to the folder you are targeting in ScanSnap Manager
2. This step captures the path of any new files dropped into the folder. It seems to catch each path individually.
3. Set a variable, based on the path
4. Gets the variable to feed into the AppleScript
5. The AppleScript accepts the variable as an {input}
   1. The AppleScript then grabs just the name of the file
   2. It hands the name of the file off to FileMaker using the GetURL command to run a script in FileMaker and pass it the File Name variable
6. We go back to the original path statement
7. Using that Path we move the file to a ‘Processed’ folder
8. Loop causes the workflow to keep going until there are no more files in the folder.

Select All and Copy the entire Automator routine into the clipboard.

Select New Automator Workflow and select the type ‘Folder Action’. Paste the script from above into the new Folder Action.

In this case, we will select the ScanSnap Folder on the Desktop.

I work with FileMaker Pro Advanced. When I deploy this solution, it will be running on FileMaker Pro. Set this line of the AppleScript accordingly.

Troubleshooting Tip: When deploying this solution, the location had a copy of FileMaker Pro 11 and FileMaker Pro 13 installed. AppleScript does not seem to have a good method of recognizing the difference between the two. My solution was to zip the FileMaker Pro 11 application to prevent it from opening.

Change the variable myDatabaseName to whatever the name of your database is.

Change the variable myDatabaseScript to whatever the name of your import script is. This script in FileMaker Pro must accept a File Name as a parameter.

What this Automator script is doing is that whenever a new file is added to the target folder (by the ScanSnap scanner, or by dropping the file in the folder), the File Name of the newly added file is handed off to FileMaker via AppleScript, using the ‘FMP’ URL Scheme. FileMaker dutifully performs the script that is called and the parameter that gets passed is the File Name.

This getURL script assumes that the database is open, logged in with the appropriate permissions to run the referenced script.

Advanced Tip: It is possible to embed a user name and password to the script, but that may be seen as a security issue. The syntax is:

FMP://[[account:password@]netaddress]/databasename
[?script=scriptname[&param=scriptparameter][&$variablename=value]]

With a user name and password, the database doesn’t even have to be open to launch FileMaker and run the script. Use hard-coded user name and passwords at your own risk.

Adding this step to the Automator Workflow is what causes the routine to run repeatedly in order to process multiple files at a time. When I first set it up, the default was for ‘Stop after 1 minute’. I noticed that the routine would import about a dozen records and then stop. By pushing the limit to 5 minutes, I was able to handle larger batches of files being dropped into the folder. If you are dropping exceptionally large numbers of files, you may need to move this limit higher. If you are using this primarily with a scanner, 5 minutes should be fine.

When an Automator script runs, a small gear icon appears on your top menu bar. To halt a runaway script, click the little x at the right of the popup box.

This is a fairly basic script, if there is a script parameter (the path), then make a new record and insert the file from the path.

1. My configuration is a target folder called ScanSnap on the Desktop. If you want to use a different name or location, edit the $filePath variable, changing ‘ScanSnap’ to ‘<<Whatever your Folder Is Called/>>’. The end slash is important.
2. Another important change is to set the Insert File script to the appropriate container field in your solution.

• Editable: If something changes in your workflow, it is fairly easy to open it up and change things.
• Flexible: When you get documents from another source than ScanSnap scanner, such as via email or Dropbox, you can still process them using the same routine, ie, drop the files in the folder.
• Tinker-able: Want to add OCR, or file compression? Open the hood, edit the AppleScript and you can build this into the routine.
• Free: Also free.

Cons

• With so many moving parts, this may be inherently breakable. Next OS X release, the entire house of cards could come crashing down. Don’t blame me. Dig into the code and fix it, then send me an update…

on run {input}
--this gets the file name from the input variable.

tell application "Finder" to set myFileName to name of first item of input

--debug: if you are having problems, turn on this display dialog 
--to see what file name you are getting.

--display dialog myFileName
--setup the database name and script name of the FMP url here:

set myDatabaseName to "HB-ScanSnap-Integration-Demo"
set myDatabaseScript to "Import_ScanSnap_Document"

try
 --change this to "FileMaker Pro" if that is your target.
tell application "FileMaker Pro Advanced"
activate

--FileMaker 13 supports this type of URL to run a script 
--inside FileMaker, passing a parameter. 
--set up your script to accept as a parameter the path of a File.

getURL "fmp://$/" & myDatabaseName & "?script=" & myDatabaseScript 
& "&param=" & myFileName
end tell
end try

--delay for a few seconds to allow the import to happen in FileMaker. 
--You may need to adjust this delay if your machine is faster or slower.

delay (3)

--go back to the Finder to process the next phase of this Automator 
--action, which is to move the processed file to another folder. 
--I called mine ScanSnap Processed.
-- you could change this last part to move the file to the trash 
--once you are sure it is working well.
-- The Loop at the end of the Automator transaction is to handle 
--when multiple files are put into the folder rapidly.

end run

The workflow in the demo is sending just the File Name as a parameter. It would make it more modular if instead it sent the entire path, then you wouldn’t have to configure the FileMaker script to reference a particular folder and you could move the folder around to different locations on your drive, without breaking the script. I tried this and it works, sort of, but for some reason instead of just inserting the file, each time it loads a document, you briefly see the Open File dialog. The end result is also that the last step leaves the Open File dialog box open, potentially confusing for the user. There may be a solution to this, but I went with what worked the first time, sending just the file name. If you want to experiment with it, here is the AppleScript code I was using.

Instead of

tell application "Finder" to set myFileName to name of first item of input

I tried:

--get the path from the value of the variable.
--set thePath to first item of input as string

--convert it from : to /
--set myFilePath to the POSIX path of thePath

--Then change
getURL "fmp://$/" & myDatabaseName & "?script=" & myDatabaseScript 
& "&param=" & myFileName

--to 

getURL "fmp://$/" & myDatabaseName & "?script=" & myDatabaseScript 
& "&param=" & myFilePath

In the FileMaker Import Document script, change the $filePath variable to Get(SystemDrive) & Get(ScriptParameter), which will be the full path starting at Users/.

According to my Googling, there are a some other solutions out there for building a paperless office with a scanner and FileMaker Pro.

Free – there is a free app called Snap2FM, which I was not able to get to work. I ended up employing pretty much the same approach as they use, using the FMP URL script protocol, but instead of compiling it into an app, I have created it as an open Automator action which is more editable.

Use a Bigger Fujitsu Scanner – Another solution is called DocKeep from Productive Computing. It starts at $199 and seems to support mostly higher-end departmental and workgroup scanners.

Use a TWAIN Capable Scanner – Combine a TWAIN scanner (Wirecutter’s second choice was this TWAIN capable Canon scanner or a slightly larger Epson scanner.) and a TWAIN plugin for FileMaker Pro. Monkey Bread Software’s do-everything MBS plugin has TWAIN capabilities. Roll your own system.

I looked for a bit for ways to do this same thing in Windows, but didn’t come up with anything solid. It is possible to send files to a Folder in the ScanSnap software on Windows. But that’s where the automation ends.

3rd Party Tools

There is a simple tool called Folder Actions, but it seems to be a one-step automator. I couldn’t see how it could do what I wanted, calling a FileMaker URL. I also looked at ‘Actions‘ which seems to be End of Life.

Microsoft

I am surprised to see that Windows still doesn’t have an equivalent to Automator built-in. I thought Redmond was good at copying things Apple did. Steve (Ballmer) would never have let this happen.

MonkeyBread/TWAIN option

The MonkeyBread/Twain solution suggested above might be a route to try. This would not work with the Fujitsu scanner, but another similar TWAIN-compliant scanner would work. This route would skip the intermediate folder and just scan directly into FileMaker.

I am not much of an Applescripter, but I do know how to copy and paste from Google searches. Here are some resources I looked at and possibly read in the process of writing this blog post.

• An overview of Automator
• Info on AppleScript and POSIX paths here.
• Using Variables in Automator
• Setting up Folder Actions here and also here at Apple’s Mac Developer Library
• Setting up the new Fujitsu ScanSnap software was not the easiest thing I’ve done. There is a decent manual, but it is long. A company called DocumentSnap has written a primer to get you rolling faster.