External libraries and dependencies

From SweepMe! Wiki
(Redirected from Device Class Dependencies)
Jump to navigation Jump to search

Not all files that you need to run your driver are part of SweepMe!, e.g.

  • Python packages that are not part of SweepMe!
  • C dll files
  • .NET dll files

Still, you can add these packages and files to your driver by loading them from a subfolder.

Please note: SweepMe! 1.5.7 and later are 64 bit applications. Python packages with compiled files and dll files must therefore be 64 bit versions. Python packages must match the Python version of SweepMe! (Python 3.9 for SweepMe! 1.5.6 to 1.5.8, Python 3.14 for SweepMe! 1.6.1 and later).

Python packages

SweepMe! ships with several essential Python packages, including:

  • numpy
  • scipy
  • matplotlib
  • pyvisa
  • pyserial
  • pythonnet
  • ...and many more (see Help & Info -> Credits for a complete list)

These packages can be used directly in your driver with standard import statements such as import ... or from ... import ....

If a required Python package is missing, you need to include it in your driver. Rather than copying the package manually into the driver folder, we recommend using the Library Builder to handle dependencies cleanly and avoid potential issues.

Note: You may only redistribute additional Python packages if their licenses permit it.

Library Builder

Copying a Python package directly into your driver may result in:

  • Missing sub-dependencies
  • Duplicated dependencies already included in SweepMe!, potentially causing version conflicts

To avoid such issues, we provide the Library Builder tool. It collects all required external packages and adds them to a libraries/ folder.

External libraries for drivers to be used with SweepMe! >=1.5.6 are created using LibraryBuilder2, which can be downloaded from the Dashboard of the SweepMe! website.

The original Library Builder (for SweepMe! 1.5.5) is no longer maintained. However, instructions can still be found in the wiki history: Library Builder.

Requirements

  • LibraryBuilder2 version 2.1.0 or later. The tool updates itself from sweep-me.net on every start.
  • Python installed in the version that matches your target SweepMe! version: Python 3.14 (64 bit) for SweepMe! 1.6.1 and later, Python 3.9 (64 bit) for SweepMe! 1.5.6 to 1.5.8.
  • External libraries must be available on PyPI
  • Internet connection (to fetch packages from PyPI and definitions from the SweepMe! server)

Instructions

  1. Start LibraryBuilder2.exe for the first time. It will generate a LibraryBuilder2.ini file next to the program and stop. Open and edit this file:
    • arch lists the bitness to build, usually [64].
    • sweepme_version is the SweepMe! version you build for, e.g. 1.6.1.1. LibraryBuilder2 downloads the list of packages that this version already ships, so these packages are not added to your driver again.
    • python_directory in the section [Environment64] is the path to your Python installation, e.g. C:\Python314. In versions before 2.1.0, this key was called python39_directory.
  2. In your driver folder (where main.py is located), create a libraries/ subfolder.
  3. Inside the libraries/ folder, create a requirements.txt file listing your dependencies.
  4. Run LibraryBuilder2.exe and specify the path to requirements.txt, or drag and drop the file onto the executable.
  5. The packages are installed into libraries/libs_<python version>_<bitness>, named after the Python installation that was used, e.g. libs_314_64 or libs_39_64. The license texts of the packages are written as LICENSE_<package>.txt files into the libraries/ folder. LibraryBuilder2 does not overwrite an existing libs_... folder, so delete the old folder before you build again.
  6. To support SweepMe! 1.5 and 1.6, run LibraryBuilder2 twice: once with Python 3.9 and sweepme_version set to your 1.5 version, and once with Python 3.14 and a 1.6 version.
  7. If you plan to publish the driver, review the licenses of all added dependencies. Packages whose license could not be determined are listed at the end of the output. Add their license files manually.
  8. If your driver supports only some Python versions, e.g. because a package is not available for Python 3.14 yet, specify this with a config.ini file.
  9. After successful execution, your driver folder should follow the structure described in Driver Structure.

Environment variables PATH and PYTHONPATH

When a driver is loaded, SweepMe! adds the folder of the driver and its subfolders libraries/libs_<python version>_<bitness> and libraries/libs_common (or the legacy folder libs) automatically to the Python path. Folders that contain dll files are also added to the environment variable PATH. pysweepme does the same when a driver is loaded with pysweepme.get_driver(). In most cases, you therefore do not need to change any paths yourself.

Convenience function

The FolderManager of pysweepme provides the function that is used for this. You can still call it yourself, e.g. in a CustomFunction script:

from pysweepme import FolderManager
FolderManager.addFolderToPATH()

The following sections describe how to add folders manually. They are mainly relevant for drivers that were created for SweepMe! 1.5.5 with the old Library Builder (libs/library.zip).

Adding a folder to PYTHONPATH

Of course, you can add folders yourself, e.g. to load a python package that comes with your driver being in the subfolder "libs", you can use

# Add the path of your libs within your driver to the variable sys.path:
import os
import sys

libpath = os.path.dirname(__file__) + os.sep + "libs"
if not libpath in sys.path:
    sys.path.append(libpath)

# Use the next three lines if the LibraryBuilder created a library.zip subfolder
libzippath = libpath + os.sep + 'library.zip'
if not libzippath in sys.path:
    sys.path.append(libzippath)

# Now import your libs as usual
import *name of the lib*

Adding a folder to PATH

In case you like to add a folder to environment variable PATH, use the following lines of code

# Add the path of your libs within your driver to the variable sys.path:
import os
import sys

libpath = os.path.dirname(__file__) + os.sep + "libs"
if not libpath in os.environ["PATH"].split(os.pathsep):
    os.environ["PATH"] = libpath + os.pathsep + os.environ["PATH"]


Adding a package that is partially shipped with SweepMe!

# In this special example, the python module 'email' shall be imported, but it is already partially part of SweepMe!. A normal "import email" would lead to an import of the module as distributed with SweepMe! that does not contain all subpackages we need. In this case, we have to change the path that is used by python to load the module by modifying sys.modules['email']. It only works if 'email' is already registered as key in sys.modules so that the property '__path__' can be modified.

libpath = os.path.dirname(__file__) + os.sep + "libs"
libzippath = libpath + os.sep + 'library.zip'
emailpath = libzippath + os.sep + 'email'                        
if not libpath in sys.modules['email'].__path__:
    sys.modules['email'].__path__.append(emailpath)

#After changing the path to 'emailpath' that is inside the 'library.zip', we can import our modules that have been created by the LibraryBuilder as usual.
import email.mime
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText

If you have problems with creating and importing modules that are not shipped with SweepMe!, please contact us.


Examples

Drivers that load additional Python packages are:

  • Logger-PC_Screenshot (libraries for Python 3.9 and 3.14)
  • Logger-PC_Midi
  • Camera-Basler_pylon

Dynamic Link libraries (DLL)

The bitness of a dll must match the bitness of SweepMe!. SweepMe! 1.5.7 and later are 64 bit applications and can only load 64 bit dll files. Loading a 32 bit dll fails with an error message like "WinError 193". Please make sure you add the correct version.

C DLL

Loading

Before you can load a C dll, one has to make sure that SweepMe! can find it. Either put the dll into the folder 'ExternalLibraries' of the public SweepMe! folder or put it into the folder of your driver. In the latter case, the path of the folder needs to be added to the PATH variable, which can automatically be done with the following two lines.

from pysweepme import FolderManager
FolderManager.addFolderToPATH()

If you add the dll file to the folder of your driver and you like to share it, please make sure that you have the right to distribute the dll file.

Imagine your dll is called 'control.dll'. You can load it via

lib = ctypes.cdll.LoadLibrary('control.dll')

or

lib = ctypes.windll.LoadLibrary('control.dll')

or

lib = ctypes.windll.control # if the dll is already known to windows

In all cases, each function returns an object 'lib' that can be used to call further functions.

lib.startMeasurement()

where 'startMeasurement' would be a function of the control.dll that is called with no arguments.

Examples

The best way to learn how to implement a C dll is to see a driver where it is already done:

  • Logger-PCsensor_HidTEMPer
  • Logger-Hamamatsu_C12918

Troubleshooting

Cannot load dll:

  • Make sure the path of the dll is added to the path variables
  • Check whether your C dll is really a C dll not a .NET dll
  • If manufacturer of your instrument also provides an interface via a .NET dll, we recommend to use the .NET dll
  • Before implementing everything as a driver, it might help to start with a simple python script that loads the dll and calls important functions. If the procedure is clear, you can start copying your code into a driver.

Problems with dll commands

  • There are two typical ways to load a C dll: ctypes.cdll.LoadLibrary(<your dll>) or ctypes.windll.LoadLibrary(<your dll>) Each variant will call the internal dll functions differently.
  • If your dll was already installed by using a program of the manufacturer of your instrument, it might be that the dll in located in a windows folder. In that you can use ctypes.windll.<name of your dll without file extension>
  • In many cases, ctypes makes sure that python objects that are used as function arguments are automatically converted to the correct C objects. However, it does not work always and it can help a lot to define the variables restype and argtypes for each function. An exemplary use for a arbitrary function 'setParameter' can be seen here:
lib.setParameter.restype  = ctypes.c_float # the type of the returned value
lib.setParameter.argtypes = [ctypes.c_float, ctypes.c_int, ctypes.c_ulong] # each item of the list refers to one argument of the function
  • It can help to already create C type objects by using ctypes, e.g.:
a = ctypes.c_float(3.14)
b = ctypes.c_int(3)
  • Often you need to create a HANDLE object. It can be done using ctypes.wintypes:
import ctypes.wintypes
my_handle = ctypes.wintypes.HANDLE()
  • Or you need to create a pointer to a certain object:
my_pointer = ctypes.Pointer(a) # a can be any ctypes object for which you need a pointer
  • Instead of creating a pointer you can also handover the pointer when you call the function using 'ctypes.byref':
lib.getData(ctypes.byref(a)) # a can be for example a ctypes buffer object, getData is just an arbitrary function
  • In contrast to python where all results are typically returned by a function, functions of a C dll sometimes would like to have a pointer to a buffer object. During the function call, the buffer is then replaced by some data and after the function call you can evaluate the originally created buffer object to read out its new content.
  • To learn how C dll function are used, check whether there is a header file, e.g. 'control.h' if your function has the name 'control.dll'. Typically, all functions are listed and described their including which arguments (and types!) they expect and which return values they have.
  • Programs like 'Dependency Walker' might help you to see all functions provided by a dll. Please make sure you have the right to do so.

.Net DLL

Loading

Before you can load a .NET dll, one has to make sure that SweepMe! can find it. Either put the dll into the folder 'ExternalLibraries' of the public SweepMe! folder or put it into the folder of your driver. In the latter case, the path of the folder needs to be added to the PATH variable, which can automatically be done with the following two lines.

from pysweepme import FolderManager
FolderManager.addFolderToPATH()

If you add the dll file to the folder of your driver and you like to share it, please make sure that you have the right to distribute the dll file.


A .NET assembly can be loaded using the pythonnet/clr package that already comes with SweepMe! with

import clr

at the top of the driver file.

With the following line, you can add the dll file to the namespace of python:

clr.AddReference('RgbDriverKit') 
# use 'RgbDriverKit' if the file has the name 'RgbDriverKit.dll'

If the PATH was not updated correctly in the first step, you get an error message like 'Unable to find assembly'.

Now, you can use the name of the added reference like a python package:

from RgbDriverKit import SimulatedSpectrometer, RgbSpectrometer, SpectrometerStatus

I some cases, the name of the dll and the name of the package that is provided by 'clr.AddReference' are not identical. Check the namespace of clr by

print(clr.__dict__)

in order to see the correct key/name that has to be used during import.

For example, a dll provided by Newport called 'Cornerstone.dll' to control monochromators can be used with the following two lines

clr.AddReference('Cornerstone')
import CornerstoneDll as cs

In this case, the name of the imported dll ('CornerstoneDll') is different from the name of the .NET assembly ('Cornerstone').

In order to see the available function of the .NET dll, you can use the tool 'ildasm.exe' that e.g. comes with Microsoft Visual Studio. Please find further information here: https://docs.microsoft.com/en-us/dotnet/framework/tools/ildasm-exe-il-disassembler

Example

The following drivers are based on .NET dll files and might help you to get started with your driver:

  • Spectrometer-RGBphotonics_Qwave
  • Spectrometer-Labsphere_CDS6x0
  • Switch-Thorlabs_K10CR1