External libraries and dependencies

Not all files that you need to run your Device Class are part of SweepMe!, e.g.

• Python packages (32bit, python 3.6) that are not part of SweepMe!
• C dll files (32bit)
• .NET files (32bit)

Still, you can add these packages and files to your Device Class by loading them from a subfolder.

Python packages

Several important python packages are already shipped with SweepMe!, e.g.

• numpy
• scipy
• matplotlib
• pyvisa
• pyserial
• pythonnet
• ... and many more, see credits.html

You can directly load them using "import ..." or "from ... import ..."

In case a python package is missing, you have to add it your your Device Class. Instead of copying the package into the Device Class folder, we recommend to use the Library Builder.

Please note that you can only redistribute python packages if the licenses of all additionally shipped packages allow it.

Library Builder

If you would simply copy a python package to your Device Class it might be that

• you are missing further sub-dependencies
• you are shipping sub-dependencies twice that are already included in SweepMe!

For that reason, we provide a tool called 'Library Builder' which adds all missing packages to a folder 'libs' which you can copy into your Device Class folder.

Download and installation

• The Library Builder is available via the download section of your sweep-me.net user account.
• You can register yourself an account using the Login button. Then, select 'Register' link.
• Unzip the package to any project folder where you have permission to write files.

Requirements

Install Python 3.6.8 32bit (i.e. same version as used by SweepMe!) with:

• pyinstaller 3.3.1 [1] (newer versions will not work)
• all packages you like to use (They have to match the python version and must be also 32bit)

Consider to create a virtual environment for that purpose.

Anaconda

Alternatively, you can use Anaconda to create the correct environment that is needed to run the library builder. Corresponding commands for the anaconda prompt are:

• SweepMe! 1.5.4: Create: "conda create -n SweepMe_154_dev python=3.6.8 pyinstaller=3.3.1 spyder", Activate "Conda activate SweepMe_154_dev"
• SweepMe! 1.5.5: Create: "conda create -n SweepMe_155_dev python=3.6.8 pyinstaller=3.3.1 spyder", Activate "Conda activate SweepMe_155_dev"

After you installed and activated the environment, you can install all further packages that are needed to import your library that is missing in SweepMe!.

Extracting the missing packages

1. Goto LibraryBuilder folder
2. Open build_library.py and change sweepme_path to your installation path of SweepMe!
3. Open required_modules.py and import all packages which are missing in SweepMe!
4. If your libs folder or the library.zip contains .pyd files and you have import errors, you can add "import pyd_importer" to the file required_modules.py It will help python to find the pyd files. However, you also need to add "import pdy_importer" to your driver before you import the actual packages.
5. Run build_library.py
6. Copy the folder libs to your device class and rename as desired
7. Do not forget to add licence files for each package that has been added
8. Also add the path to your libs folder to the environmental path variables so that SweepMe! can find and import it.

Environment variables PATH and PYTHONPATH

Often, you have to add the folder and subfolders of a Device Class or CustomFunction function to the environment variable PATH or to PYTHONPATH to allow for importing libraries and dlls (dynamic link libraries). We created a convenience function that adds all folders and subfolders of your DeviceClass script to both environment variables, so that all further imports should work immediately.

Convenience function

The 'FolderManager' of SweepMe! provides a convenience function that automatically adds the folder and all subfolders of the script to PATH and PYTHONPATH. Just add the following two lines of code to your DeviceClass before you import external libraries:

from FolderManager import addFolderToPATH
addFolderToPATH()

Adding a folder to PYTHONPATH

Of course, you can add folders yourself, e.g. to load a python package that comes with your Device Class being in the subfolder "libs", you can use

# Add the path of your libs within your Device Class to the variable sys.path:
import os
import sys

libpath = os.path.dirname(__file__) + os.sep + "libs"
if not libpath in sys.path:
    sys.path.append(libpath)

# Use the next three lines if the LibraryBuilder created a library.zip subfolder
libzippath = libpath + os.sep + 'library.zip'
if not libzippath in sys.path:
    sys.path.append(libzippath)

# Now import your libs as usual
import *name of the lib*

Adding a folder to PATH

In case you like to add a folder to environment variable PATH, use the following lines of code

# Add the path of your libs within your Device Class to the variable sys.path:
import os
import sys

libpath = os.path.dirname(__file__) + os.sep + "libs"
if not libpath in os.environ["PATH"].split(os.pathsep):
    os.environ["PATH"] = libpath + os.pathsep + os.environ["PATH"]

Adding a package that is partially shipped with SweepMe!

# In this special example, the python module 'email' shall be imported, but it is already partially part of SweepMe!. A normal "import email" would lead to an import of the module as distributed with SweepMe! that does not contain all subpackages we need. In this case, we have to change the path that is used by python to load the module by modifying sys.modules['email']. It only works if 'email' is already registered as key in sys.modules so that the property '__path__' can be modified.

libpath = os.path.dirname(__file__) + os.sep + "libs"
libzippath = libpath + os.sep + 'library.zip'
emailpath = libzippath + os.sep + 'email'                        
if not libpath in sys.modules['email'].__path__:
    sys.modules['email'].__path__.append(emailpath)

#After changing the path to 'emailpath' that is inside the 'library.zip', we can import our modules that have been created by the LibraryBuilder as usual.
import email.mime
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText

If you have problems with creating and importing modules that are not shipped with SweepMe!, please contact us.

Examples

Device Classes that load additional python packages are:

• Camera-PC_Webcam
• Logger-PC_Midi
• Logger-PC_Screenshot
• Switch-PC_Winsound

Dynamic Link libraries (DLL)

SweepMe! can only load 32bit dll files. Please make sure you add the correct version.

C DLL

version 1.5.4

Loading

Before you can load a C dll, one has to make sure that SweepMe! can find it. Either put the dll into the folder 'ExternalLibraries' of the public SweepMe! folder or put it into the folder of your Device Class. In the latter case, the path of the folder needs to be added to the PATH variable, which can automatically be done with the following two lines.

import FolderManager
FolderManager.addFolderToPATH()

If you add the dll file to the folder of your Device Class and you like to share it, please make sure that you have the right to distribute the dll file.

Imagine your dll is called 'control.dll'. You can load it via

lib = ctypes.cdll.LoadLibrary('control.dll')

or

lib = ctypes.windll.LoadLibrary('control.dll')

or

lib = ctypes.windll.control # if the dll is already known to windows

In all cases, each function returns an object 'lib' that can be used to call further functions.

lib.startMeasurement()

where 'startMeasurement' would be a function of the control.dll that is called with no arguments.

Examples

The best way to learn how to implement a C dll is to see a Device Class where it is already done:

• Logger-PCsensor_HidTEMPer
• Spectrometer-InstrumentSystems_CAS140CT

Troubleshooting

Cannot load dll:

• Make sure the path of the dll is added to the path variables
• Check whether your C dll is really a C dll not a .NET dll
• If manufacturer of your instrument also provides an interface via a .NET dll, we recommend to use the .NET dll
• Before implementing everything as a Device Class, it might help to start with a simple python script that loads the dll and calls important functions. If the procedure is clear, you can start copying your code into a Device Class.

Problems with dll commands

• There are two typical ways to load a C dll: ctypes.cdll.LoadLibrary(<your dll>) or ctypes.windll.LoadLibrary(<your dll>) Each variant will call the internal dll functions differently.
• If your dll was already installed by using a program of the manufacturer of your instrument, it might be that the dll in located in a windows folder. In that you can use ctypes.windll.<name of your dll without file extension>
• In many cases, ctypes makes sure that python objects that are used as function arguments are automatically converted to the correct C objects. However, it does not work always and it can help a lot to define the variables restype and argtypes for each function. An exemplary use for a arbitrary function 'setParameter' can be seen here:

lib.setParameter.restype  = ctypes.c_float # the type of the returned value
lib.setParameter.argtypes = [ctypes.c_float, ctypes.c_int, ctypes.c_ulong] # each item of the list refers to one argument of the function

• It can help to already create C type objects by using ctypes, e.g.:

a = ctypes.c_float(3.14)
b = ctypes.c_int(3)

• Often you need to create a HANDLE object. It can be done using ctypes.wintypes:

import ctypes.wintypes
my_handle = ctypes.wintypes.HANDLE()

• Or you need to create a pointer to a certain object:

my_pointer = ctypes.Pointer(a) # a can be any ctypes object for which you need a pointer

• Instead of creating a pointer you can also handover the pointer when you call the function using 'ctypes.byref':

lib.getData(ctypes.byref(a)) # a can be for example a ctypes buffer object, getData is just an arbitrary function

• In contrast to python where all results are typically returned by a function, functions of a C dll sometimes would like to have a pointer to a buffer object. During the function call, the buffer is then replaced by some data and after the function call you can evaluate the originally created buffer object to read out its new content.
• To learn how C dll function are used, check whether there is a header file, e.g. 'control.h' if your function has the name 'control.dll'. Typically, all functions are listed and described their including which arguments (and types!) they expect and which return values they have.
• Programs like 'Dependency Walker' might help you to see all functions provided by a dll. Please make sure you have the right to do so.

.Net DLL

version: 1.5.3

Loading

Before you can load a .NET dll, one has to make sure that SweepMe! can find it. Either put the dll into the folder 'ExternalLibraries' of the public SweepMe! folder or put it into the folder of your Device Class. In the latter case, the path of the folder needs to be added to the PATH variable, which can automatically be done with the following two lines.

import FolderManager
FolderManager.addFolderToPATH()

If you add the dll file to the folder of your Device Class and you like to share it, please make sure that you have the right to distribute the dll file.

A .NET assembly can be loaded using the pythonnet/clr package that already comes with SweepMe! with

import clr

at the top of the Device Class file.

With the following line, you can add the dll file to the namespace of python:

clr.AddReference('RgbDriverKit') 
# use 'RgbDriverKit' if the file has the name 'RgbDriverKit.dll'

If the PATH was not updated correctly in the first step, you get an error message like 'Unable to find assembly'.

Now, you can use the name of the added reference like a python package:

from RgbDriverKit import SimulatedSpectrometer, RgbSpectrometer, SpectrometerStatus

I some cases, the name of the dll and the name of the package that is provided by 'clr.AddReference' are not identical. Check the namespace of clr by

print(clr.__dict__)

in order to see the correct key/name that has to be used during import.

For example, a dll provided by Newport called 'Cornerstone.dll' to control monochromators can be used with the following two lines

clr.AddReference('Cornerstone')
import CornerstoneDll as cs

In this case, the name of the imported dll ('CornerstoneDll') is different from the name of the .NET assembly ('Cornerstone').

In order to see the available function of the .NET dll, you can use the tool 'ildasm.exe' that e.g. comes with Microsoft Visual Studio. Please find further information here: https://docs.microsoft.com/en-us/dotnet/framework/tools/ildasm-exe-il-disassembler

Example

The following Device Classes are based on .NET dll and might help you to get startet with your Device class

• Spectrometer-RGBphotonics_Qwave
• Spectrometer-Labsphere_CDS6x0
• Camera-Basler_ace