Skip to content

drmcnelson/TCD1304-Sensor-Device-with-Linear-Response-and-16-Bit-Differential-ADC

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

403 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

TCD1304 Sensor with Linear Response and 16 Bit Differential ADC


By Dr M. C. Nelson, Copyright 2020-2026, Patent Pending

Important Build Note: To ensure performance and alignment with the validated hardware and firmware, please build only from the Original Repository.

For technical assistance or commercial inquiries, please contact me. I do my best to answer emails as volume allows.

This repository provides the open-sourced hardware, firmware and documentation for a low noise high-precision Linear CCD instrument. The present 2026 upgrade introduces a hardware-locked timing architecture utilizing the i.MX RT1062's FlexPWM that provides enhanced thermal and electrical stability and strong attenuation of charge transfer residuals (ghosting). The resulting system achieves <0.2% Integral Non-Linearity (INL) over essentially the full dynamic range of the sensor and exposure range from 10 μsec and above. Additionally the system maintains these performance specs with radiometric accuracy across high-gradient spectral transitions.

System Performance and Validation

The following table summarizes the performance metrics achieved in the present design which features a physics-informed electrical architecture and hardware-locked timing system. This instrumentation focused approach prioritizes metrological stability and the elimination of electronic artifacts at the detector interface. The system utilizes a dual-stage differential front-end (AD4807 and THS4521) specifically tuned to ensure signal settling to 16-bit precision ($&lt;$ 0.0015% error) within the constraints of the CCD's charge-transfer physics. By maintaining a 30:1 slew rate margin (225V/μs capability vs. 7.5V/μs demand) and electrical noise below 1 LSB, the design ensures that the variances observed in our Photon Transfer Curve (PTC) methodology are a reflection of sensor shot noise and silicon characteristics, rather than an artifact of the readout electronics.

Validated Performance & Metrological Characteristics

Metric Validated Value Characterization / Methodology
Integral Non-Linearity (INL) < 0.2% Measured to >99% of well capacity in phase-locked pulse loop mode, preserved across high gradient spectral transitions.
Quantization 16-bit (65,536 ADU) External differential 1MSPS ADC (MCP33131D) with low impedance FDA driving circuit.
Read Noise Floor (σ) 52.2e-, temporal stability σ=6.8e- Derived from photon transfer curve (PTC) analysis over an 810-frame dark ensemble with pairwise subtraction to isolate Fixed Pattern Noise (FPN).
Gain (K) 0.812e-/ADU, uniformity σ=0.0755e-/ADU Calculated via the inverse slope from the PTC method.
Timing Stability (Jitter) < 10 ns Hardware-locked synchronization using i.MX RT1062 FlexPWM and PIT modules.
Dynamic Range (Ensemble) > 140,000:1 103 dB ratio of saturation limit to characterized noise floor; validated via PTC.
Dynamic Range (Single) > 5000:1 Instantaneous range sufficient for high-contrast absorption and fluorescence studies.

The following figures provide representative validation of the system's metrological integrity, demonstrating the high spatial uniformity and linear response achieved through hardware-locked timing.

Systematic Error (PLM Mode)

Residual error remains within a narrow band up to 99% of full-well capacity (0.19% INL).

Gain Distribution (PIT Mode)

Spatial response is highly uniform across 3,648 pixels (σ = 0.0755 e⁻/ADU).

Comparative Benchmarking: A "Stare-Down" Test

Stability and linearity are the physical prerequisites for reproducibility in scientific CCD instrumentation. The validation data above—specifically the 0.19% INL and the 0.075 e−/ADU spatial uniformity—establishes the baseline for this system. A primary test of this metrological integrity is the normalization of spectra by exposure time; in a truly linear system, these normalized plots will overlay perfectly. Because our hardware-locked architecture eliminates the 'sag' and baseline drift common in commercial drivers, we achieve a common intercept across the entire dynamic range. We invite researchers to perform these same 'stare-down' tests with commercial spectrometers to see the difference in deterministic response.

(a) Fluorescent lamp spectra normalized to exposure time exhibit consistent intensity and baseline. (b) Peak heights are linear in exposure time to near saturation.


Background and Project Mission

This repository offers a linear-CCD sensor system based on the TCD1304DG, designed specifically for stable, reproducible, and linear response. In precision spectrometry, stable linear response is a physical prerequisite for reproducibility. If an instrument fails to maintain fidelity at the hardware level, the resulting data is fundamentally compromised.

Achieving this requires a holistic, science-centric design that insists on metrological integrity at the source. Our approach addresses the foundational physics of the sensor and the signal chain simultaneously, ensuring that the instrument is responsive to real spectral line shapes, high-gradient imaging features, and holographic fringes.

The Challenge: "Black Box" Instrumentation

Perhaps surprisingly, foundational issues—non-linearity, slew-rate limitations, and baseline instability—often go unaddressed in commercial hardware. These artifacts are typically handled in ways including:

  • Numerical Patching: Using post-readout mathematical "corrections" to hide hardware flaws.

  • Log-Transform Dismissal: Claiming linearity is secondary because data is eventually log-transformed into absorption units.

We believe that relying on software and numerical slight-of-hand to obscure physical non-linearity is unacceptable for a scientific instrument. As evidenced by our comparisons with entry-level commercial units (e.g., Ocean Optics), a lack of baseline stability and poor slew/settling management leads to spectral smearing and radiometric drift. This repository provides a "Radiometrically Honest" alternative where the data reflects physical reality, not a software estimation.

A Definitive Design for the TCD1304

Since their inception in the late 1980s, CCD spectrometers have promised a low-cost, "all-at-once" spectral capability. However, as many of us who have worked with these sensors for decades are aware, they have historically been plagued by baseline instability and "ghosting" (residual charge transfer).

The goal of this project was to finally and fully resolve these issues by utilizing a hardware-locked timing architecture (via the i.MX RT1062 FlexPWM) and a ground up design for the analog front end, pulse driver system and power architecture. This definitive design provides a level of confidence in linearity and reproducibility that allows researchers to publish their data with confidence. Our aim is to "set a new bar" for what can be achieved with open-source scientific instrumentation.

The SPI Instrumentation Project - Open Instruments for Open Science

The contents of this repo are part of our effort in Open Instrumentation for Open Science.

We believe that access to doing great science should not be limited to those privileged in funding and affiliation nor held ransom to the extractive practices of the market in scientific instruments. And anyway, you may feel better served by instruments developed by other scientists who actually use them and have decades of experience designing professional instrumentation. The designs we provide can often be built at about 1/10 of the cost of the commercial instrument. In our own research, we are typically able to do more with these boards than we can with the expensive commercial instruments.

At this writing, we have received requests and helped scientists working in Europe, Africa, India, Canada and the USA.

One very important way that you can help underfunded scientists is by clicking the "Sponsor" button at the top of this repo. The funds go to helping to make more instruments more available to more scientists around the world.

If you would like to sponsor or receive boards, please contact me.

Permissions, no warranty or guarantee, and etc.

Permission is hereby granted for you to build and use these boards and codes in your lab and for your personal use. Please cite appropriately if you use these in your published work.

Portions of the hardware-locked timing architecture and other aspects of the design, are subject to pending patent protection.

Please contact me if you need/want:

  • Pre-assembled boards
  • Customization, advice, etc.
  • Permission for use in a product or other commercial effort

And of course, no warranty or guarantee is given whatsoever. We did our best.

If you have questions, please feel free to contact me. And, don't forget to click the "Sponsor" button (or contact me directly).

Hardware Strategy

The hardware architecture is engineered to provide reliable operation and high-precision linear transfer of spectra and holographic frames while isolating the system from electrical and mechanical noise.

  • Differential Analog Front-End (AFE): A fully differential signal path utilizing the ADA4807 and THS4521 ensures maximum common-mode rejection. The AFE is significantly over-specified for slew rate and bandwidth, ensuring that sharp features in spectra and images settle completely within the sampling window.
  • High-Current Gate Drivers: To manage the significant capacitive load of the TCD1304 (notably the ~600pF SH gate), the design utilizes dedicated 50mA pulse drivers. This ensures sharp edge transitions critical to meeting the timing requirements of the sensor and achieving stable radiometric performance.
  • Power & Grounding Topology:
    • Separate Planes: The PCB utilizes dedicated power and ground planes, with separate LDOs for analog and digital domains.
    • Tuned Decoupling: Power rail decoupling was modeled and optimized in SPICE to effectively dampen commutation transients and attenuate "push-through" noise on the supply rails.
    • Return Path Logic: The pulse ground return is specifically designed to minimize common-mode impedance coupling and ground-bounce artifacts.
  • Precision Timing Engine (FlexPWM): Hardware timing is driven by the i.MXRT’s 150 MHz FlexPWM modules. Unlike standard MCU timers, this allows for nanosecond-scale, phase-locked control over the CCD logic. As noted, precision timing is critical to stable radiometric performance.
  • High-Performance Compute: The system utilizes the NXP i.MXRT1062 (ARM Cortex-M7) running at 600 MHz with 480 Mbps USB 2.0 (High-Speed). This provides the computational and communications headroom to support the maximum frame rate of the TCD1304 without impacting the deterministic nature of the hardware timing.
  • Physical & Mechanical Isolation:
    • Two-Card Architecture: The sensor, AFE, pulse drivers, and ADC reside on a dedicated instrument card, separate from the MCU.
    • Cable Strain Isolation: This isolates the MCU from the analog subsystems and ensures that the mechanical stress of connecting USB or synchronization cables to the Teensy does not translate to the sensor card, preserving optical alignment.

Firmware Strategy - Asynchronous State Machine

The 2026 timing architecture utilizes three independent but coordinated submodules (plus a fourth as the master clock) of a single FlexPWM on the i.MXRT1062 to manage charge transfer and readout. This approach replaces sequential software loops with a phase-locked hardware state machine.

Three Logic Engines are implemented in the interrupt states of the three submodules.

  • SH (Shift Gate): Manages idle, exposure start and end state transitions.

  • ICG (Integration Clear Gate): Preparation and transition to readout.

  • CNVST (Conversion Start): A dedicated machine with idle and readout states.

Strategic use of latencies and timing windows for the above preserves critical timings in gate and ADC operations, both of which are important in noise and linearity performance. The CNVST idle acts as a virtual buffer for the ADC reference voltage system. The SH idle runs on a fast clock to ensure residual charge clearance in the sensor subsystem. The read engine implements a zero copy system that leaves data frames on the ring buffer for asynchronous transfer to the host computer.

Metrological Validation

The efficacy of the hardware-locked timing and differential front-end is supported by the following characterization data.

  • Linearity and Dynamic Range: The system maintains an Integral Non-Linearity (INL) of <0.2% over five orders of magnitude (10 μsec to 0.5 sec) in exposure and up to 95% of the sensor's physical saturation ceiling. Radiometric accuracy is preserved across high-gradient spectral transitions over the full dynamic range.
  • Noise Floor & Signal Integrity: Characterization of the AD4807/THS4521 front-end confirms an electronic noise floor of ~1 LSB (quantization limited) isolated from the sensor. With the TCD1304 integrated, the total system noise floor is ~0.6 mV. Pairwise frame subtraction in PTC analysis confirms residual variance is dominated by sensor noise rather than electronic artifacts.
  • Charge Transfer Integrity: Hardware-locked SH idling is validated by direct methods. However, it is also seen simply comparing PIT and PLM operating modes. Activating the PIT clearing pulse engine results in a decisive intensity drop and brings measurements into alignment with PLM benchmarks thus confirming effective flushing of the shift register.

Repository Contents

This project is part of a larger mission to provide Open Instruments for Open Science, supporting researchers and under-resourced labs globally. This repository contains the complete ecosystem required to build and validate the system:

  • Hardware: Fab files and BOMs for three versions: 16-bit Differential ADC (high-end), 12-bit, and Analog-out.

  • Firmware: An Arduino-compatible "sketch" and a high-performance header-only C++ library.

  • Host Software: Python-based CLI and class libraries with real-time graphics and offline data processing.

  • Validation: Technical tutorials (this document) for optical/electrical design and a collection of SPICE models used to test and refine the signal chain.

  • Performance Benchmarks: Documented results showing linear, reproducible response over 5 orders of magnitude in exposure time (from 10 μs).

Implementations

We provide three implementations of the sensor system hardware (see the following figures); (a) a two board implementation comprising the sensor board and our Teensy 4 based instrument controller, (b) a single board "All-In-One" implementation with sensor and MCU on back and front of the same board, and (c) an Analog board with the sensor, signal conditioning circuit and gate drivers with analog output of the inverted and amplified sensor signal.

In the following we provide a high level description of each of the three implementations For each we also describe the cost and choice to build or buy. The costs listed with each implementation include the sensor and microcontroller which are currently running at $40 and $24 respectively and the PCB which generally runs around $18 per board in small quantities (including tariffs).

Note that the performance metrics achieved by the design are a function of both the hardware and firmware.

Two board system, 16 bit sensor board and controller

The high end sensor system, shown here, is a two board system comprising sensor board and controller. It offers very low electrical noise with a 16 bit 1MSPS ADC and good mechanical isolation of the sensor from the controller. The ribbon cable carries logic signals and power for the SPI interface (1.7V-5.1V). The two wire connection (red and black) is 5V power. Internally, there are separate low noise power circuits and ground planes for the analog section and gate drivers. We observe 0.6mV dark noise, electrical noise is more than 10 times lower (less than 1 LSB with the sensor removed). And the board is able to linearly follow peaks to full scale in one pixel. Fiduciary marks on both sides of the sensor board facilitate optical alignment.


TCD1304 Sensor system, (a) sensor board bottom showing sensor and fiduciary marks, (b) sensor board top showing circuits, interconnects and baseline trim, and (c) sensor board and controller showing geometry and interconnection (control and data plus 5V power).

Component costs for the high end 16-bit system are currently $110 for the sensor board and $88 for the controller, or $198 for the set, plus the time it takes to do the assembly work. The passives are generally 0603, some are 0402 and two of the ICs are 0.5mm pitch. It takes us a few hours per board for hand assembly, or about one day per board set.

We recently switched to using a PCBA service for the SMT parts. Normally this would bring our costs to $290. With tariffs our cost per set is now $395 to $422 depending on the clearance agent. We feel that compared to hand assembly it is still a bargain.

"All-in-one", sensor and controller on a single board.

The following shows the single board "all-in-one" device with sensor, electronics and controller all on one board. This device still offers good performance in terms of linearity but with 12 bit precision. It has a single ended analog signal path and uses the built-in analog input of the Teensy 4.0 (and therefore has fewer parts, costs less and is easier to hand assemble).


TCD1304 All-In-One Board, (a) bottom showing the sensor, (b) top showing the microcontroller board and pins for auxiliary functions, and (c) view showing the overall geometry.

The component costs are currently $86 including TCD1304 and Teensy, plus $18 for the PCB, for a total of $104. We generally assemble these in house. The passives are 0603 or larger. The two IC's are 8 pin, 0.65mm pitch. It takes us a few hours or about half of a day.

If you use this with the provided firmware, in TCD1304Device2c.h look for and uncomment the line "// #define ALLINONEBOARD". This configures the code to use the internal analog input of the microcontroller rather than the external ADC.

Analog sensor board with gate drivers

The following shows our analog-output sensor board. This also has the single ended analog circuit as in the all-in-one board, and similar gate drivers. The board can be powered from 4V to 5.5V and accepts 3.3V to 5V logic to operate the CCD gates. We developed this board to provide an actually useful alternative to the analog boards offered on some DIY sites. The output is intended to be compatible with the inputs of typical processor boards in the Arduino ecosystem, but linearity and ability to meet the clocking requirements for the CCD sensor will depend on which Arduino ecosystem board and firmware you choose to use. Running the board from our Teensy 4.0 controller and firmware, we found that it has very good linearity including for sharp spectral lines.

 

TCD1304 Analog Board, (a) top showing the circuits and connectors, (b) bottom showing the sensor.

Parts costs are currently $65 including the TCD1304, plus $18 for the PCB per the above, for a total of $83. The passives are 0603 and the ICs are SOT23 packages to make it a little easier for hand assembly. It takes us perhaps 3 hours to build.

On linearity and reproducibility in spectrometers

As noted, reproducibility is vitally important for any instrument and for a spectrometer stability and linearity are pre-requisite for reproducibility (as well as for basic capabilities such as signal averaging). CCD spectrometers are historically challenged by these criteria. We discuss this in the section titled Linearity and Reproducibility in CCD Spectrometers. Included following are results of our "stare-down" comparison in data collected side-by-side from the present design and a widely used commercial instrument. The data illustrate the relationship between linearity, reproducibility and being able to produce meaningful data.

Construction of the spectrometer used for testing

Construction of the spectrometer used for testing the new sensor is described below (here). We use a 1200/mm grating and 200μm entrance slit with a focal length of 2 1/4". Total cost of materials for the spectrometer is under $450, including the electronics (this repo), optics and mechanical parts.

Controller

As noted, we operate the sensor board using our newly updated Instrumentation Controller based on the Teensy 4, with its NXP i.MXRT 1060 ARM7 MCU (please click here). The T4/NXP platform is somewhat unique among MCU boards in the Arduino ecosystem for its speed at 600MHZ, its high speed USB at 480Mhz and its multi-channel FlexPWM module. The latter is critical to the precise timing and functionality needed to achieve high-end performance metrics and operation over five orders of magnitude in exposure time.

Firmware

The firmware (here), written for the T4, includes a header-only library to operate the sensor, and a "sketch" file (Arduino source code, similar to C++) that implements human readable commands and responses, operates the sensor to produce frames by clock or hardware trigger, and sends the data back to a host computer.

We expect that the firmware provides all of the functionality you may want for almost any sort of experiment. However, we provide the TCD1304 library and the complete source code so that you can modify it if you wish. We work with the code using the Arduino IDE for compilation and Emacs as an external editor.

Python user interface with graphical display

The Python code directory (here) provides a program file TCD1304Controller.py that can function as a command line user interface with real time graphical display or as a class library to operate the instrument and save data to ASCII disk files, a program file DataReader.py that similarly can be used as a utility program or class library to work with and graph the saved data in 2-d or 3-d, and a collection of bash shell scripts that use DataReader.py to generate graphs of various sorts.

The software design emphasizes flexible simplicity and performance, again with anticipation that scientist users can modify the Python code to their purposes. The command "help" lists detailed help text from the controller and from the Python code.

Electronic design

A tutorial on electrical design for CCD sensors and spectroscopy is included here. The section begins with the datasheet and characteristics of the sensor, proceeds to signal condition and driving an ADC and finishes with a section on the gate drivers. SPICE files are included in a subdirectory of the repo. You can modify the SPICE files to investigate your own designs.

Fab files (Gerber, BOM) and codes

The fab files and code provided in this repo, and in the controller repo, plus some cabling and a host computer, should be sufficient to assemble the boards and get your detector system up and running. Feel free to contact me for consultation or pre-assembled boards (time permitting). And needless to say, donations are very much appreciated, please find and "click" the sponsorship button above.


High-level description of the hardware-firmware-software architecture

The sensor is operated through four input pins, a clock and two gates plus a fourth for the "convert" pin of the ADC, and an SPI interface which retrieves the data from the ADC. These connect to the controller and the controller in turn connects to the host as a serial port device over USB. The controller also provides a trigger input and sync output and additional pins that can be used to interact with other equipment.

The controller operates the sensor device with its FlexPWM module programmed to serve as timing generator for the clock, gates and ADC convert signals, and its SPI module to read the ADC. A small set of interrupt service routines maintain frame counters, measure exposure times, and so forth. At the end of the readout, the data record and the various parameters that accompany the data record are organized into a C struct as a "frame" and put onto a ring buffer. In the main thread, the function loop() checks the ring buffer, sends pending data frames to the host, and processes command inputs from the host computer. Every command is acknowledged by sending "DONE" back to the host.

The Python code running in the host, represents the sensor device and its controller through a class object, TCD1304CONTROLLER. A multiprocessing thread TCD1304port listens to the port to receive data frames and messages sent in response to commands, and interacts with the main thread through a set of queues; the data queue, a queue for commands to send to the controller and a graphics queue for real time display. The graphics window runs in a separate thread also.

Thus we have two levels of buffering, one in the controller and one in the host software, and commands and data are serialized on both ends of the interconnection between the host and controller. The commands and responses are all simple human readable ASCII. Data can be transferred as binary or ASCII.

The sensor board internals include the TCD1304DG sensor, gate drivers, analog signal conditioning and ADC. The external interface comprises the three gate inputs, an input to command the ADC and SPI to retrieve the data.


Achieving linearity and reproducibility is to a large extent a function of hardware and firmware. The analog subsystem and ADC provide linear response to spectroscopic signals, the gate drivers enable effective retrieval of charge and the firmware provides effective operation.


Getting it all up and running

This sensor board is intended to be used with our new Teensy 4 (T4) based controller. The files provided here (gerbers and code) and in the controller repo, plus some trivial cabling and a host computer (we recommend Linux for the best frame rate perfomance) should be sufficient to build and operate the boards.

If you are using the "All-In-One", then you have the TCD1304 and circuitry and the Teensy 4.0 all on one module. The gerbers are provided in their own directory in this repo. There is one macro switch in the firmware that you need to uncomment to compile for the "All-In-One" board.

Assembling or Obtaining boards

You can assemble the boards yourself, or if you prefer, please feel free to contact me for pre-assembled boards.

If you want to assemble your boards, and this is your first time assembling an SMT board, search for an introduction to DIY SMT assembly, for example here.

Here are some notes on how we do assembly in our shop. We order PCBs from AllPCB, JPLPCB, or PCBWay. We usually order parts from Digikey, but we also use Mouser and Newark. We use Chip Quik no-clean solder paste in a syringe dispenser with fine needle tips that we order separately. And we use a reflow oven that we purchased through ebay for about $200, and sometimes we use a temperature controlled rework heating stage that we purchased through Amazon.

USB connection

We recommend using a powered USB hub with switches to turn individual USB devices off and on. When you shop for this, make sure it supports high-speed (at least USB2.0) and can supply at least 1A per port. For example, a powered 7-port USB hub should be able to supply at least 1A x 5V x 7 ports = 35W.

Loading the firmware

After the boards are assembled, you will need to install the Teensy board in the controller, and compile and load the code into the Teensy. You will most likely want to use the Arduino IDE for this. Teensy is well integrated into the IDE. See here for setup instructions. The Teensy needs to be connected by USB to your host computer for this step.

The firmware codes are found in the repo in the Firmware subdirectory

(In the following, the "260419" in the directory and file name, is the date of this version of the firmware. If there is a newer version when you read this, use that one.)

TCD1304Device_Controller_260419/

  TCD1304Device_Controller_260419.ino  - The controller program with CLI

  TCD1304Device2c.h  -  A header-only C++ library for the TCD1304 and Teensy4.x (iMXRT106x)
  
  parselib.cpp      - A string parsing library for the CLI
  parselib.h

  Doxyfile          - Doxygen configuration file

The Arduino IDE requires that the "ino" file and directory have the same name.

To load firmware into the "All-In-One" board, open the file TCD1304Device2c.h in a text editor or find its "tab" in the Arduino IDE, and un-comment the following line. It is at line 24 in the current version of the file.

  //#define ALLINONEBOARD

If you want to customize the firmware, it is recommended to create a new directory, copy the files to that directory and rename the ino file per the above.

After installing the Arduino IDE and Teensy package, you should be able to double click on the ino file to start an IDE session, or start the IDE and navigate to the directory and open the file.

Setting up and running the Python codes

Python codes and Bash scripts for operating the TCD1304DG sensor and working with the data, all under Linux, are provided in the repo under the subdirectory Python/.

The codes have been used with the Fedora Cinnamon Spin, which you can download from here. This uses xorg rather than Wayland, the desktop is Win7-like and it is easy to work with terminal windows. We have also done some preliminary testing in Ubuntu.

Installing the Python environment

The command to install the Python environment and libraries used by the codes is as follows (in Fedora, use apt-get or aptitude in Ubuntu):

$ sudo dnf install python python-numpy python-scipy python-matplotlib python-pyserial

Setting up the user software for the TCD1304 boards

To setup the Python codes from this repo, unpack or download the files from the repo's Python subdirectory to a directory on your Linux machine; somewhere under your personal user directory works just fine. And, set the permissions to allow execute (chmod a+x *.py, and chmod a+x *.sh).

Here is a list of the files provided in the Python directory

TCD1304Controller.py   - User interface and Class library

GraphicsWindow.py      - Libraries used by TCD1304Controller.py
GUIWindow.py
TextWindow.py

DataReader.py          - Offline library and graphics utility

SetupPATH.sh           - Adds the code directory to PATH
    
GraphTCD130Spectrum.sh - Offline graphics using DataReader.py

Calibration2           - example wavelength calibration

Running the user command interface with real-time display and post collection shell scripts

You will want to add the directory for the Python and bash scripts to your PATH. You can do this by adding the following line to your .bashrc. Alternatively you can run this from a command terminal, but you would then need to do it each time. Note that the command begins with ". "

$ . pythoncodesdirectory/SetupPATH.sh

After all of the above, make sure that your sensor controller is connected to your compute with a USB cable, that the cables are connected correctly if you are using the two board implementation, and then turn on the power and wait about 1/2 minute.

Now you can run the controller program.

$ TCD1304Controller.py

The controller should open a grahics window. The desktop will look something like this.

Notice that in the console window, we have a prompt. This is the command line interface presented by the Python program. The Python CLI provides commands to wait, save to disk, run commands from a file, run shell commands, and etc., and passes all other commands to the hardware. The command help, produces a listing of the commands recognized by the hardware and Python CLIs. A listing of the help output can be found in the repo here. A summary of some of the most often used commands can be found at the bottom of this readme.

The firmware provides commands at three levels. The high level commands include the following to collect clocked or triggered data.

   tcd1304cli> read <n frames> <exposure>
   tcd1304cli> read <n frames> <exposure> <frame interval>
   
   tcd1304cli> trigger <n frames> <exposure>
   tcd1304cli> trigger <n frames> <exposure> <frame interval>

The first form collects "back-to-back" frames with exposure time congruent with the frame interval. This accommodates intervals as short as 8 to 20 msecs depending on the master clock period (see the command "set clock ..."). We refer to this a "PIT mode" or "single pulse mode". Dark noise has a minimum at about 10-20 msecs. There is no effective upper limit on exposure time in this mode, apart from the increase in registering cosmic rays. Signal averaging can be done on line (see add) or after data is save to a file.

The second form collects fast "frame sets" with short exposure times using a timing architecture we refer to as "pulse loop mode". The exposure time in this mode can be as short as 10 μsecs depending on pulse widths. The frame interval needs to be at least the readout time plus the exposure time, c.f. 10msec for a 1msec exposure with the master clock period at 0.6usec or faster (see the command "set clock ..."). Signal averaging is available for this mode, too.

The trigger input can be configured as follows, where <option> can be any of rising, falling or change, pullup or nopullup, or pin <pin-number>.

   tcd1304> configure trigger <option>

For kinetic studies using back to back exposures, the following command can be used to configure the pulse sequence to run "cleaning pulses" before each exposure. You can read more about this here. The 260419 firmware provides a second parameter for the pulse period, at least 5usec is recommended. The pulse loop architecture described above takes care of clearing pulses internally.

   tcd1304cli> configure clearing pulses <n> [<period(secs)>]

Middle level commands including setup pulse.., setup pulse loop..., setup timer, start and trigger, provide data collection capabilities with detailed control of the timing for the pulse sequence that operates the sensor. There is also a complete set of low level commands for register level access to the FlexPWM timing generator in the MCU.

The Python controller program saves incoming data onto a queue. The command save <filespec> retrieves and writes the data to disk. The command clear empties the data queue without writing to disk. The saved data includes the "0" frame. The first exposure interval is frame 1.

Data frames can be added using add all which sums all of the data into one frame, or add all after n which sums all the frames after the first "n" frames, or add ensemble(s) which sums the data at each index in the frame set. After adding the frames, you can use save as above, or collect more data and add again. (See the next section "On Linearity and Reproducibility...")

Following is an example that shows the data produced with a single frame at low intensity compared to that produced by adding 100 frames collected at the same signal intensity. The signal to noise ratio increases by a factor of 10 as expected (√N). Aside, being able to add data frames and obtain meaningful data is possible only when your instrument is linear. We will discuss this in further detail in the next section.

The program and class library DataReader.py can be used to work with the offline data. The command line accepts python language statements and can produce graphs. Setting variables x, y, y2, yscatter, etc., generates a 2-d graph. Setting surface or image generates a 3d surface or heatmap.

The following example generates a simple 2-d graph of electron counts divided by exposure time. Omitting --output sends the graph to the screen. More examples are included in the bash scripts in the distribution.

   bash# DataReader.py mydatafile \
           d=dataset[0] \
           x=d.xdata \
           y="d.frames[-1].dataCounts()/d.exposure" \
           xlabel="\"Wavelength(nm)\"" \
           ylabel="\"Counts/sec\"" \
           --output mydatafile.png

To list all of the available variable names and data associated with a file,

   bash# DataReader.py mydatafile --dump

The data files are in ASCII and human readable. If you wish, you can work with the files using a spreadsheet program. The advantage of DataReader.py, whether you use it from the command line or as a library, is that it parses the file into Class objects and you can use the full power of the Python libraries to work with the data.


On Linearity and reproducibility in CCD spectrometers (with data)

In this section we discuss linearity and reproducibility, in a practical sense, as related to CCD spectrometers. After defining terms and concepts, we show data comparing the present design and a widely used commercial instrument. These illustrate the basic concepts and establish in a small way the "facts on the ground".

Linear response for a spectrometer means that the measured response S is proportional to the number of photons P impinging on the detector. For a change in intensity at pixel "n", we expect that ΔSn = cn ΔPn where cn is a constant.

When a system is linear we should see that (a) spectra collected with different exposure times agree with each other (S1/t1 = S2/t2), (b) ratios of peak heights are constant (Sλa/Sλb at t1 = Sλa/Sλb at t2), and when summed the result agrees with that obtained by a single measurement with the combined exposure time S = St1 + St2 = St1+t2.

Notably our linearity criterion was expressed as a change in P and S. Normally we would apply the above rules after subtracting a noise or background signal. Conveniently, for this kind of sensor, the dark noise SD is proportional to exposure time in the range of exposure times greater than 20msec and for this sensor system the electrical noise is several orders of magnitude smaller than the dark noise. Therefore, for these longer exposure times, the total intensity (S = SP + SD) should also be linear.

That said, there are a few ways in which spectrometer response can be non-linear. Some of these can be corrected numerically provided the non-linearity meets certain mathematical criteria. For example, measured values should at least be monotonically increasing in exposure time so that there can exist a unique mapping between a measurement and its corrected value.

However, some non-linearities involve bandwidth or line shape. And while a valid correction might exist, it is most often far easier and far more reliable to start with an instrument that has linear response.

So far we have talked about linear reproducible response to the signal produced by the sensor. We will next show some examples, give mention to another class of behaviours associated with the physics of moving charge within the sensor, and then finish with some further discussion of linearity and electrical characteristics of signals in spectroscopy. Detailed discussion of anomalies associated with charge distribution and effects of timing on linearity are included at the end of the README beginning with the section on gate drivers and analog integrity.

Let's look at some data.

Spectra

The following are fluorescent lamp spectra, from the present design and from a commercially produced spectrometer. Notice that the line at 436nm in the new instrument is about twice the height of the 546nm line whereas in the commercial instrument it is attenuated by a factor of 4. The lines at 436nm and 546nm correspond to well known lines of Hg. Their tabulated intensity ratio is approximately 2:1, similar to that obtained with the new instrument (a) ("Strong lines of Mercury", NIST).


Fluorescent lamp spectrum, (a) new sensor and (b) commercial instrument.

This seems like a simple test. But some care is appropriate. One way to alter the spectrum is to simply misalign the sensor. But then the effect should be systematic. In this instance the intensities of the other lines in the commercial instrument seem inconsistent with the large difference in the 436nm line.

The following shows the spectrum from the new instrument with the y axis expanded so that we can see the structure in the region around 590nm. We see that the lines are a little sharper compared to the commercial instrument.


Spectrum from the new instrument with expanded intensity scale to show detail around 590nm and 615nm.

Intensity

The following shows the raw intensities versus exposure time for four of the peaks that appear in the above spectra from the present design and from the commercial instrument. We select the strongest two lines, at 435nm and 546nm, and the smaller peak at 542nm and the wider peak at 487nm.

In a linear instrument, all of these intensities should rise linearly with exposure time or overall intensity. For the new sensor (a), the curves are indeed straight lines from near the origin until near saturation. For the commercial instrument, most of the range is not linear. We will see more explicitly how this effects relative peak heights.


Intensity versus exposure time for four spectral lines for (a) the present design and (b) the commercial instrument. The present design demonstrates linear response.

Consistency and practical reproducibility

A simple test for consistency and practical reproducibility is to collect spectra at a different exposure times or with the light source at a few different distances with source and aperture aligned on a rail. Either of these methods will vary the number of photons and hence the number of electronics registered in each pixel. When the sensor system is linear, we should be able to scale by exposure time (or R2) and obtain the identical spectrum apart from the difference in signal to noise ratio.

Here is the result varying exposure time for the new sensor (the present design) and for the commercial instrument. As we noted in the introduction, the spectra produced by the new sensor are nearly identical apart from noise.


Spectra normalized to exposure time for the (a) the present design and (b) the commercial instrument.

For the present design, spectra are demonstrated to be linear, reproducible and consistent across a wide range in intensity. For the commercial the spectra are clearly not consistent. Perhaps, if we could reproduce exposure conditions exactly we might see the same spectrum twice but it would have no meaning. For the commercial instrument we have no feasible solution for the inverse problem to convert it data back to any sort of radiometric truth. To be able to say something that is meaningful, we need to have data that is linear, reproducible and consistent over a realistic range of experimental conditions. We refer to this criterion as practical reproducibility.

Peak height ratios

We reasonably expect that in a reliable instrument ratios of intensity should not change when we change intensity or exposure time. First, we expect that spectra should have the same appearance in terms of how large one peak is compared to another when we repeat a measurement. Second, quantitative comparison of intensities is a basic element of many experimental protocols.

The following set of figures shows ratios of peak heights as a function of exposure time. The present design shows roughly constant peak height ratios until one of the peaks in the ratio reaches saturation.


Peak height ratios for the lines at 546nm and 542nm versus exposure time in (a) the present design and (b) the commercial instrument.


Peak height ratios for the lines at 435nm and 546nm versus exposure time in (a) the present design and (b) the commercial instrument. The correct ratio is close to 2:1 with the line at 435nm the larger.

Some further phenomena

The above are examples of non-linearity in the conventional sense of linear response of a circuit and digitizer to an analog signal. There are two further issues that can effect linearity in a different way.

Baseline integrity

Baseline or background subtraction is often a necessary step in extracting intensity data from spectra. There are a number of ways to do this, for example using dark spectra or regions of spectra where the experiment produces little intensity. The former assumes the background is independent of the signal of interest and the latter assumes background is dominated by the dark noise of the detector rather than light.

The following shows a fluorescent lamp spectrum from Wikipedia.

Notice the anomalous baseline to the blue side of the sharp peak at 631nm. The shape is not like dark noise nor any sort of room lighting. The important point about this for the present discussion is that it is not easily corrected.

Fluorescent lamp spectrum, HR2000

Fluorescent lamp spectrum.
Original: Deglr6328 at English WikipediaDerivative work: H Padleckas, CC BY-SA 3.0,
via Wikimedia Commons

Residual image (carry-over)

In studies of dynamic phenomenon, we are interested in the intensity registered in the detector during the time of a particular exposure. In CCD detectors there is always some residual charge that is carried over to the next frame. The magnitude of this carry-over effect depends on how the shift gate is driven. A detailed discussion is included at the end of the section on Residual Charge and Mitigation.

The Time-Domain Challenge: An Introduction to CCD Readout

An important point for the present discussion is that the CCD sensor records a discrete pattern of light in space (or wavelength), and on readout, this spatial map is converted into a discrete series of voltages in time. Accordingly, a sharp spectral line—an abrupt spatial gradient—becomes a high-speed pulse in the time domain.

This transformation is what separates spectroscopy from general signal acquisition. While an audio or RF circuit might tolerate slight phase shifts or non-linearities near its Nyquist limit, a CCD spectrometer must render a full-scale voltage step from one pixel to the next with absolute fidelity. If the analog front-end lacks the bandwidth or slew rate to settle within that pixel's timing window, the 'wavelength' data is irrecoverably smeared.

The following provides an introduction to how a spectrum or image is acquired from a CCD detector. Later we describe how the detector works in more detail and connect the several aspects of its operation to measures needed to reduce noise and achieve good linearity.

The following depicts a circuit model for our simplified linear CCD, an array of pixels each compromising a photodiode and capacitor connected by a switch to one element of an analog shift register.

CCD Readout

(From an archived application note, Toshiba)

The following shows the structure of a pixel in our simplified notional CCD, pink indicates n-doping. The shift gate (SH) opens a channel and biases the shift register to harvest charge from the photodiode. The electrode labeled φ belongs to the readout register.

CCD Readout

The following depicts the process for moving charge along the readout register (in the direction orthogonal to the plane of the above diagram). At the completion of each clock cycle (typically 2 or 4 clock pulses) charge has been shifted one step to the right. The charge at the last element is converted to a voltage and presented at the output.

CCD Readout

An important point for the present discussion is that the CCD records a discrete patten of light in space (or wavelength) and on readout this pattern becomes a discrete series of voltages in time. Accordingly, a sharp spectral line becomes a short pulse in time. This is what makes spectroscopy different from other signal acquisition scenarios.

What does this mean for circuit design?

In designing circuits for acoustics or radio frequency work, we might think in terms of a Nyquist frequency and we might accept some small non-linearity for signals approaching this "cutoff". But in a CCD spectrometer (or imaging system) a full scale step in voltage from one sample to the next can be a legitimate feature that has to be rendered to a meaningful digital representation. We can think of this in terms of bandwidth (units of 1/t) or in terms of dV/dt (units of V/t).

The following graph shows the Fourier transform of the above spectrum (blue) and the response curve (orange, y2 axis) for a simple single pole low pass filter with cutoff frequency at 1/2 of the sample rate. Naive filtering of this sort results in about 10% attenuation for the high frequency components that may be important for linear response to narrow spectral lines.

CCD Readout

Fourier transform of the fluorescent lamp spectrum (blue) and single pole f/2 filter (orange).

Now let's look at dV/dt. Here we graph the spectrum as dV/dt versus time (as read from the sensor). The spectral line at 436nm that is markedly stronger in the present design instrument also has the largest dV/dt. In electronics, dV/dt is related to slew.

Fl Lamp Specrtum, dV/dt at ADC

First derivative (dV/dt) of the fluorescent lamp spectrum.

There are a number of ways in which circuits can be slew-limited. Current-starving the sample and hold capacitor in an ADC is perhaps one of the more popular methods. Choosing an OPAMP with too small a maximum slew is another. And a well-abused emitter follow is a third method that seems popular in the DIY ecosystem. As we will see the difference in cost to do this correctly is small.

For the purposes of a scientific instrument, true linearity must be maintained across the full range of observable line shapes—from broad fluorescence to nearly-delta-function atomic lines. Achieving this requires more than just high-speed components; it requires a design that accounts for the dynamic load of the ADC and the specific current-sourcing needs of the signal path. We discuss the hardware implementation and the critical role of slew-rate margins in the Section on Electrical Design.

However, raw speed is only half of the equation. Even with a perfect analog front-end, the internal device physics of the CCD—specifically the transport of charge through the silicon lattice—introduces subtle non-linearities and 'memory effects' on a finer scale. To achieve metrology-grade results, these detector-level artifacts must be mitigated through precise timing and thermal stability. We take up these subjects in the final sections of this documentation, after establishing the necessary electrical prerequisites.


Setup for linearity testing

The equipment list for our linearity study is as follows. Construction of the spectrometer is described below.

  1. Spectrometer
  2. Fluorescent lamp
  3. Neutral density wheel filter for attentuation (individual filters can be used instead)
  4. 200μm optical fiber
  5. Miscellaneous mechanicals to hold the lamp, ND filter and fiber.

Linearity Measurements

Once set up and aligned, the mechanical configuration remains fixed through the duration of the measurements. The ND filter wheel is adjusted and left in a fixed setting for each dataset, each comprising a set of exposure settings.

Spectrometer Design and Construction

The following describes a simple approach to designing a spectrometer, and in particular the instrument that we used to test the new sensor device. We offer a formalism in which the choice of lenses and grating emerge in a simple way from the choice of spectral range and the size and pixel density of the sensor.

For our design, we use a transmission grating rather than a reflecting grating and folded geometry. The transmission geometry is simpler and provides good performance with reasonable cost. The following pictures show (a) the inside of the instrument, sometimes referred to as the "optical bench" and (b) the cover which is constructed of black opaque plastic. The base is aluminum plate. The sensor can be seen mounted after the second lens and the controller can be seen at the top rear of the cover with a blue USB cable running to the computer. For the present design we chose a center wavelength at 500nm. The wavelength range is 450nm. Optical resolution is about 0.5nm with a 50um slit.

Spectrometer Assembly   Spectrometer Assembly

The parts list for the above is:

  1. Grating, 1200 grooves/mm, Thorlabs GT50-12, $250
  2. 200μm entrance slit, DIY-Optics.com, ~$30
  3. Plano Convex lenses (50mm to 70mm bfl), ebay, ~$20
  4. SMA905 fitting, Amazon, Digikey, Mouser, Ebay ~$15
  5. Aluminum plate, OnlineMetals.com or Amazon
  6. Mounts produced with a 3-d printer
  7. Opaque black plastic sheets and black PVC tape, Amazon
  8. TCD1304 sensor board and controller from this repo, with cables

Design of a CCD Spectrometer

The following diagram shows the transmission geometry. Two lenses sandwich a grating with an aperture at the focal point of the input and the sensor at the focal point of the output and oriented parallel to the grating.

A productive way to think of this is that the optical system images the aperture onto the sensor, the grating transforms wavelength to angle, and the sensor samples the resulting spatial pattern.

The "master equation" and an essential criterion

If there can be said to be a "master equation" for a CCD spectrometer, it would likely be the following

δλ/Δλ = M wslit/LD

wherein the ratio of spectral resolution δλ to range Δλ, is equal to that of the widths of the slit wslit and detector LD multiplied by the optical magnification M.

The optical magnification contains a factor for the geometry multiplying the ratio of the two focal lengths,

M = (cos(θin)/cos(θout)) x (LF/LC)

For a CCD detector, we have a fixed number of pixels Npixels with which to sample our spectra. Quite obviously it serves little use to have a peak of width δλ occupying less than a few pixels. In practice a good choice is 5 pixels. Therefore we want that

δλ/Δλ > 5/Npixels

For the TCD1304 we have 3648 active pixels in approximately 30nm of length. Therefore we want Δλ/δλ < 729.

Example:

For 0.5nm resolution, our spectral range can be as large as 364.8nm. To accomplish that with a 50um slit we need a magnification of 0.82.

Center wavelength and geometry

We noted in the above that the magnification term is simply the normal magnification (ratio of focal lengths) multiplied by a term for the geometry as the ratio of the two angles.

The geometry follows from our choice of the center wavelength, and our choice of grating, through the following equation,

G λ0 = sin θin + sin θout.

where G is the line density for our grating.

Example:

For G = 1200 l/mm and a center wavelength of 500nm, we find that the sum of the sines is 0.6. If we set the exit angle to 0, then we have θin = 37 degrees. (A 1200 lines/mm, 500 nm glass transmission grating will usually be blazed at 37 degrees. See Thorlabs or eBay.)

Focal lengths, range and resolution

The spectral range to be covered by the instrument is a function of the geometry, the size of the detector and the focal length of the output (focusing) lens,

G Δλ = cos(θout) LD/LF

Example:

We want a range of 364.8nm. Therefore the right focusing lens will have a focal length LF = 1 x 30mm/(1200 l/mm x 364.8mm) ≈ 68mm.

The spectral resolution is a function of the geometry, the size of aperture, and the focal length of the input (collimating) lens,

G δλ = cos(θin) wslit / LC

Example:

We want a spectral resolution of 0.5nm with a 50um slit. Therefore the right collimating lens will have a focal length LC = 50um x 0.8/(1200 l/mm x 0.5nm) ≈ 66mm.

Notice that our angles and focal lengths agree with our expected magnification factor, (0.8/1) x (68/66) = 0.82

Diffraction limit

One more thing to check is that we are not asking for resolution that is better than our diffraction limit. For the focus the diffraction limit is

δλdiff = 1.03 λ0 M Δλ / 2 LD tan(NA)

and for the grating we have

δλdiff = 0.84 λ0 cos(θin) / 2 G LC tan(NA)

where NA = n θ is the numerical aperture, n is the index of refraction and θ ≈ λ0/wslit is the angular spread at the entrance and focus spot. Both of the diffraction limit formulae reduce to approximately δλ/2. In other words, for this instrument δλdiff ≤ δλ/2.

That completes the design, for our purposes.

Construction

After settling on the parameters and digging through our box of lenses and gratings and aluminum plate, and perhaps ordering a few thing from ebay or Thorlabs, it is time to build the spectrometer. We start with our Al plate and draw two lines intersecting at the selected angle for the incoming and outgoing optical axes. The vertex is where the grating will be mounted. Then holes are drilled to mount the lenses, each at about 20 to 30 mm from that vertex to leave space for the grating housing. We 3-d printed the mounts for the grating and lenses. Then, mounting one lens at a time, we use a flash light to find the location of the focus of each lens along its line. That is where the aperture and sensor will be located. Note that the sensor is actually 0.7mm behind the face of its window. We drill the remaining mounting holes, install everything, apply black tape to dampen stray reflections, and connect the cables to the sensor. We tweak the location of the sensor while monitoring the spectrum with the software with a fluorescent lamp as input. Now we cover the instrument with a non reflective case, save a fluorescent spectrum and calculate the coefficients to convert pixel number to wavelength and load them into the instrument (see the firmware command "coefficients").



Electrical design

We now describe some of the elements of circuit design for a CCD based spectrometer (or imaging system). This will be something of a tutorial. The idea is to support open science. That includes being open about what you need to know to design something like this for yourself. We assume some basic knowledge of electronics. The "attentive reader" will note that we make extensive use of SPICE models. A set of these is provided in the SPICE subdirectory. These form the basis of many of the illustrations shown in this README.

We start with characteristics of the TCD1304DG and then proceed through signal conditioning, interfacing to the ADC, the gate drivers, and the power architecture. Our description of the sensor are initially at a level sufficient to develop the basic electrical design. Later we will revisit these devices in more detail, drawing from a classic paper in the literature of the CCD detector, and develop the final touch that gets us to metrologic performance with $\lt 0.2\%$ INL. However for that we need some context in the basics for reading and operating a CCD and interfacing to an ADC.

TCD1304DG electrical characteristics and operation

The TCD1304DG datasheet can be downloaded here. The device architecture is shown in the following figure found on page 2 and labeled "Circuit Diagram". We see a photodiode array, shift gate (SH), integration clear gate (ICG), CCD analog shift register and signal output buffer (OS), with pins SH, ICG, master clock ΦM, and OS plus power and ground.

TCD1304 internal diagram

TCD1304DG "circuit diagram", from page 2 of the datasheet.

Operation of the device is shown in the following timing diagrams which can be found on pages 6 and 7,

TCD1304 internal diagram

TCD1304DG timing diagrams, (a) coherent shutter and exposure, (b) "electronic shutter function", from page 6 and 7 of the datasheet.

As depicted, exposure begins and ends on the trailing end of pulses asserted on the SH pin, readout begins following assertion of SH and ICG together, and thereafter, data is clocked out at 1 pixel per four cycles of the master clock ΦM.

A table of capacitances for these pins is found on page 5. Without going into the details of how CCDs are constructed (see here), we can infer that the large capacitance of the SH and ICG pins are consistent with these playing an important role in harvesting charge in the device. Four clock cycles per pixel readout further indicates a 4 cycle type CCD analog shift register.

We discuss how to drive the gates, and in particular the shift gate here. The present discussion focuses on the analog signal conditioning part of the design.

TCD1304 internal diagram

TCD1304DG "gate capacitance", from page 5 of the datasheet.

Electrical characteristics of the TCD1304 output are described on page 3 of the datasheet (excerpted here). We include the diagram from "Note 8" to indicate the origin and direction of the signal in volts. The output is a negative going signal from fixed DC level VOS typically 2.5 volts (var 1.5V to 3.5V) to VOS -0.6V at saturation (min -0.450V) and the output impedance is typically 500Ω (max 1kΩ).

TCD1304 output characteristics
TCD1304 internal digram

TCD1304DG noise

The TCD1304DG datasheet reports dark noise as 2mV (typ, max 5mv) with 10 ms exposure. The following shows our measurement of the dark noise as a function of exposure time using our 16 bit sensor board. As expected, dark noise is linear in exposure time. But there appears to be a floor for dark noise at about 0.6mV, which it approaches for exposure times less than 20ms.

TCD1304 noise

Dark noise vs exposure time in the new sensor device. (Electrical noise with sensor removed is ~14uV)

Removing the chip from the board, the electrical noise is about 14uV. So it seems that the dark noise signal at 0.6mV really does originate in the sensor chip.

Dark noise is known to be related to temperature with the usual Boltzmann dependence. Some have reported reductions in noise by a factor of 4 with modest cooling to around 4C.

Signal conditioning

For best performance we want to match the output of the TCD1304DG to the input range of our ADC. For the two board configuration, this will be a 16 bit high precision ADC with a differential input and 4V reference. For the "all-in-one" board this will be the 12 bit analog input of the MCU with a range of 0 to 3.3V. In either case we need to shift, flip and amplify the output from the sensor to match the input range of the ADC.

TCD1304 internal digram

Single ended signal conditioning

The following shows a reasonable approach for a simple shift, flip and amplify while accommodating the wide variation in output impedance of the sensor. We use a dual OPAMP, the ADA4807-2, slew 225μV/s, input noise 3.1nV/√Hz, 0.1pA/√Hz, and input capacitance 1pf. The first unit is configured as a voltage follower to take care of the large variation in source impedance and the second is setup as an inverting amplifier with offset. This gives us reproducible gain and it provides linear response with good noise performance. We use this approach for our 12 bit systems including the "all-in-one" and analog boards.

CCD signal conditioning

Low noise differential signal conditioning

We use the following approach for 16 bit precision. Similar to the above, the first unit is configured as a follower but the second is configured as an inverter with offset and unity gain. The two outputs together provide a differential signal. In implementation we follow this with a fully differential amplifier (FDA) and a differential input ADC. For gain we get a factor of 2 for free and make up the rest in the FDA. Cancellation between the differential pair improves noise performance for our mixed signal environment.

CCD signal conditioning

A don't-do DIY circuit

The following shows a design that appears from time to time in DIY postings. The "inventor" typically omits the voltage-follower and instead goes straight to the inverting amplifier. This of course makes the sensor part of the gain equation, G = R2/(R1+Rsensor). But the sensor impedance as we noted above, varies from 500Ω to 1kΩ. If the inventor is aware of the issue, they might make R1 very large to drown out the contribution from the sensor. But to have gain, R2 has to be even larger, typically 2 to 5 times R1. Now come the problems.

CCD signal conditioning off.

With large values for R1 and R2 there is a large parallel resistance that dominates the noise density at the input, vn ≈ 0.13 √R// [units nV/√Hz] (see "Johnson noise"). This creates a trade-off between bandwidth and precision. And with a very large R2, the pole formed with the input capacitance of the OPAMP at fp = 1/(2πR2Cinp) moves to lower frequency and can be within the bandwidth needed for readout. The amplifier may be unstable and the data unreliable. All of this is for a net savings of about $3 for leaving out the voltage-follower. If you need to report spectra with reproducible intensities, it might be best to avoid devices that take this approach.

Another don't-do circuit

This is another circuit that shows up in the DIY ecosystem. It is a basic transistor circuit that can be found in the first chapters of many electronics textbooks. Intermediate level text may treat the non-linearities and other limitations of the circuit including those related to trying to use it to drive an analog input. Here we will try to explain in a simple way why this is not a good idea for a spectrometer.

The following shows the bare emitter follower in two flavors, on the left with a PNP transistor and on the right with an NPN. Notice the output is taken at the emitter.

CCD signal conditioning

PNP and NPN followers.

Provided conditions are such that the base-emitter diode is "on", the output can be thought of as being maintained at one diode drop above (below) the input,

Vout = Vin +/- VBE

where VBE is the base-emitter voltage. The basic non-linearity is that VBE is not a constant, but is instead a function of emitter current and temperature

VBE = VTln(IE/IS)

and the emitter current IE varies with the output voltage,

IE = (VEE - Vout)/RE

The quantities VT for the thermal voltage (\~25meV @RT) and IS for the saturation current (10-15-10-12A), are both temperature dependent. All of this comes to about 1-2\% variation in gain, if not for the next issue.

The more serious issue, and classic for the emitter follower, arises in the proviso that the base-emitter is "on". Consider the following in which we model the emitter follower with 1k resistors for the emitter resistor and load. Note that without the transistor, the voltage at the node connecting R1 and R2 would be 1/2 of the supply voltage. We can operate the emitter follower to this voltage and no further. Below this voltage current flows into the emitter. When the output reaches this voltage the emitter current is zero. Increasing the input voltage only further reverse biases the diode.

NP follower Rload

PNP follower with resistive load.

The situation is a little more complicated with a reactive component, c.f. a capacitor, in the load and more so with the switched capacitor input of an ADC. Typical behaviors are summarized in the following.

PNP and NPN with switched capacitor

PNP and NPN followers driving the ADC sampling capacitor.

The PNP(NPN) works very well as a current sink(source), i.e. for the falling(rising) side of the waveform. On the other side, the maximum slew rate is current limited. Provided the current required ΔV/RL is less than the quiescent emitter current, the maximum slew rate in this part of the response is

(dV/dt)max = IE/CL = (VEE - Vout)/RECL

For larger ΔV/RL, the slew rate drops further to

(dV/dt)max = (VEE - Vout)/(RE+RL)CL

Recall that accuracy in an ADC depends on the sampling capacitor being able to reach the input voltage before the end of the sampling window in time. It is therefore easy to see that current limited slew is something that needs to be considered carefully. Let's consider some examples.

For 16 bits of accuracy and a 0.5usec sampling window, we need a maximum slew of at least 16 x ln(2) x 0.6V/0.5usec = 14V/usec (or 10V/usec for 12 bits). In practice we should use a circuit with a significantly larger maximum slew to avoid the roll-off region.

Lets consider a 16 bit ADC, with CL = 30pf, RL = 1K and VEE = 3.3V from a low noise LDO rather than the USB power line. Our maximum slew with the emitter follower is 6V/usec. But, with a 12 bit ADC 10pf and 2K, the maximum slew would be 10V/usec. If we go for 10 bits, then we might avoid the roll-off region for slew. The issue with that is that there is still the non-linearity in the response and the precision is not sufficient for the dynamic range of the detector.

So, the approach with a single transistor follower is marginal at best for these parameters. There are ways to make it better, but at that point you will have come close to reinventing the opamp. The cost for a dual ADA4807 which is rail-to-rail, and very linear and has a slew rate of 225V/usec, is only $5.

Interfacing to an ADC

The present application requires analog to digital conversion at rates from 200KSPS to 1MSPS and between 12 and 16 bit precision depending on your specific needs. This put us in the domain of the SAR type ADC (successive approximation register) see here. There are some important details to using a SAR type ADC and moreso for our application. This involves some nuance, so we start from the basics.

Spice model for ADC input

The SAR architecture comprises a sample and hold (S-H) circuit followed by a comparator and DAC which are operated by internal logic to implement the successive approximation algorithm. The S-H circuit is seen by the driving circuitry as the input to the ADC. In a simplified sense, it can be modeled like the following but with the switch driven by a clock.


We implement this in a SPICE model as follows. For purposes of illustration, we include an ideal voltage source as the input. C1 is the sampling capacitor, S1 is the switch that connects C1 to the input, and S1 is controlled by the clock V1. When S1 opens, the voltage on C1 is converted to a digital representation by the SAR engine. The time during which S1 is closed, is called the sampling window. (Values for the internal resistance (Ron) and capacitance (C1) for a given ADC are usually available in its datasheet.)


ADC model with ideal voltage source.
Green = input, blue = sampling capacitor, grey = sampling window.

Factors effecting precision

For n bits of precision and full scale range Vfs, the voltage on C1 at the end of the sampling window has to be within 1 part in 2n of Vfs. The input always has some series resistance, in this case Ron. Therefore the sampling window has to be at least as long as n x ln(2) x Ron x C1.

Voltage noise of a capacitor is vc = √(kT/C). For n bits of precision, we need vc < Vfs/2n. The 30 pF sampling cap shown in the model produces about 11μV of noise and 1/216 = 15μV.

When you select an ADC, make sure to look for these parameters in the table of electrical characteristics or the equivalent circuit in the datasheet. Also don't forget to look at the graphs for SNR. Often the SNR quoted in the beginning of the datasheet is less than the whole story. And don't forget to look at PSRR. And do follow the guidelines for selecting the voltage reference and for layout.

ADC kickback

Kickback describes what happens when the switch closes to connect the input to the driving circuit.

In the following we drive the S-H from a voltage follower instead of the ideal voltage source. Now when S1 opens or closes we see spikes (circled) on both voltages and in the current through R1. This is the famous "ADC kickback". It arises in the current needed by C1 with the sudden change in voltage when the switch closes.


ADC model driven by OPAMP follower.
Green = out, blue = sar cap, grey = sampling window, red = current through R1.

Charge reservoir as solution for kickback

In the often recommended solution for kickback, we provide a charge reservoir in the form of a capacitor in front of the input to the switched capacitor network, as in the following illustration. Now, when the switch closes, current flows from the charge reservoir, which is replenished on a somewhat slower time scale by the driver (the opamp).


The following shows the implementation of this scheme in our SPICE model. A cap C2 is added in front of the ADC to act as the charge reservoir for C1. We see in the traces that charge for C1 now comes from C2 and the voltage on C2 is managed by a much smaller current through R1. There is no discernible kickback in V(out) nor in the current through R1.


ADC model with recommended driver, charge reservoir C2.
Green = out, turquoise = adc in, blue = sar cap, grey = sampling window, red = current through R1, orange = current through C2.

The charge reservoir and series resistor together look like a low pass filter, but the components are chosen a little differently. C2 needs to be large compared to C1 and R1 needs to be tuned between the current capacity of the OPAMP and allowing C2 to track the input. We make extensive use of SPICE modeling to check designs for response and precision.

Spectral line meets charge reservoir

You may recall our discussion about dV/dt and sharp spectra lines near the top of the README. Here is what happens when a large dV/dt meets the charge reservoir that we added to support the sampling capacitor in the ADC. The circuit in this example is a "full feature" single ended model from CCD to ADC. The input is a pulse, and a little bit extreme with a 2ns rise time.

Notice that we now have a large current through R1 coincident with the leading and trailing edge of the pulse. The current through C2 is as before. So, large dV/dt meets cap serving as charge reservoir and produces a new kind of kickback.

Effect of dV/dt in charge reservoir.
Green = out, turquoise = sar cap, blue = sar cap, grey = sampling window, maroon = current through C2, red = current through R1.

The following shows a method for mitigating the new kickback. We simply slow the pulse a little bit with a low pass filter in the feedback loop. This has to be tuned somewhat to put the current within the capabilities of the OPAMP but still allow C2 to recover fast enough to meet the design requirements for precision. We are able to get 18 bit precision in models. Real spectra are less demanding and our precision spec is 16 bits.


SPICE model for the 16-bit analog front-end and ADC Input

In the preceding sections, we discussed some general concepts about signal conditioning and interfacing to an ADC. We briefly mentioned that we use a differential signal path to achieve low noise in our mixed signal environment. We have the analog signal from the sensor alongside the gate and clock signals for the sensor, with the analog and voltage reference input to the ADC and its digital interface to the microcontroller. In this environment, the differential signal path is effectively a pre-requisite for meeting our performance requirements in noise and linearity.

The following show our SPICE model for our analog signal path and ADC input (follow the link to download the LTSpiceTM file). This the circuit used in the 16 bit sensor board. For purposes of "worst case" modeling, we setup the input signal as a short pulse. Calculating and then hand tuning the values for the passives in the charge reservoir system and the FDA feedback loop, we are able to manage the kickback while still achieving convergence to better than 1 part in 16 bits.

SPICE model for the 16 bit sensor board, analog section.
Red = sensor, Green = adc inp, turquoise = Cs+ (sar cap), blue = current through R11.

Convergence of V(Cs+) to input voltage for the 16 bit sensor board.
Red = sensor, Green = adc inp, turquoise = Cs+ (sar cap).

Gate driver and analog signal integrity

Issues that require careful attention in designing the gate drivers, include the fact of the very large capacitance of the SH gate, the role of this gate in collection charge from the photodectors into the readout CCD register, and then an electrical effect in the rest of the circuit wherein noise from the gate drivers appears on the power rails.

Current requirements for driving the gates

Recalling the table at the top of the discussion on electrical design, we saw that the shift pin has a capacitance of 600pF. As we will see in a moment, the shift pulse needs to be driven with a rise time that is short compared to the duration of the shift pulse, generally a factor of 20 can be appropriate. This means that for a 1 usec shift pulse, we need at least 4V x 600 pf/ 50 nsec = 48mA.

A suitable way to drive the gate would look like the following. We use a SN74LVC1T45 and an 82ohm series resistor.

Mitigating noise generated on the power rails

The following shows a SPICE model for the gate drivers. These generate a pulse on their power rails, so we power these from a separate voltage regulator. The model includes a current limited supply for the LDO with inductor, in place of the filtered current limited 5V supply on the actual board.

Gate drivers and pulse generation on the power rails.
Blue = V(sh), Green is V(icg), Red = (V(ccb)-4.0492) x 1000, Grey = I(R6)

Note that the trace for the voltage pulse on the supply side of the gate drivers is scaled times 1,000. Using this model we confirm that the amplitude of the pulse is well within our power supply noise budget for the analog signal path.

Residual Charge Effects and Mitigation

Now lets take a look at another way in which the gate driver effects performance in the analog section. The issue we are going to describe now is largely mitigated away by our timing architecture. However we do provide some level of control, and in any case it is useful to understand something of how this works.

In our earlier discussion of linearity we briefly described the architecture of a simplified notional pixel comprising a photo-detector region and an element of the analog shift register. In an approximate sense, we have a three step process; (1) light produces charge in the photo-detector region, (2) pulsing the shift gate moves charge to the shift register, which is then (3) shifted away by clocking the shift register. But, some charge necessarily remains behind in the photo-detector region. The quantity depends on material properties, dimensions and temperature and the voltage and duration of the pulse applied to the shift gate.

Charge collected in the photodetector region is moved to the analog shift register by applying a voltage pulse to the shift gate.

Let's see at what this effect can look like in practice. The following figure is collected with a timing generator coded into a second Teensy board. We use this second board to generate a frame trigger for our spectrometer with the LED turned on for every other frame. For these data we use the spectrometer design described above with our 16 bit board and Teensy 4.0 controller, These are 20 msec exposures with the shift gate driven at 4V for 1 usec for one pulse per frame. Whether the LED is on for 17 msec or 4msec, we obtain a similar residual charge in the frame after the LED is off. In other words, for both exposure levels, the first pulse on the shift gate removes the majority of the charge collected in the photo-detector and leaves us with a similar level of residual charge for the next frame.

Spectra with LED on (blue curve) for (a) 17 msec or (b) 4 msec, each immediately followed by a second frame with LED off (orange curve). All frames are 20 msec exposures with no clearing pulses. The single shift gate pulse at the end of the exposure removes most of the charge and leaves a similar level of residual charge at about 1000 ADU (gain ~ 1e-/ADU).


The following figure shows the residual carry-over as a function of the width of the pulse applied to the shift gate at each of several voltages. This data was collected using our analog board to allow us to vary the voltage applied to the shift gate. We show the fit to an exponential for each voltage. The floor and exponential follow voltage. The time constant of the driving circuit is 600pf x 100 ohm = 60nsec for the analog board. We see that at 4V the residual charge plateaus by 1usec.

Residual charge is reduced with increasing shift gate pulse width and voltage.

Based on our conceptual model for the pixel we might expect the residual charge floor to follow a "Boltzmann" like curve in the applied voltage. The next figure shows the floor from the above study versus the voltage of the shift gate pulse. We see that it does indeed follow a simple exponential in voltage. Perhaps fortuitously, the coefficient is approximately the well depth and the "rate" constant is about 1V.

The residual charge floor following a single pulse versus shift gate voltage with fit to a Boltzmann.

So far we see that the first pulse ending the exposure removes most of the charge collected in the photodector leaving a small residual charge. We know that under dark conditions we eventually see only baseline, i.e. dark noise. In other words, while the charge collected in the photo-detector region is sufficiently large redistribution under the applied shift gate field is fast. The self-field evidently helps remove the charge. But then at small levels of charge we are left with a slower process. Nonetheless, it is still diffusion under an applied field. We can derive a simple model for that process as follows.

We start with a residual charge $q_0$​. With the next shift gate (SH) pulse, a fraction R of that initial charge is successfully transferred ("Readout") to the analog shift register. The remaining fraction (1−R) stays in the photodetector region.

  • Charge successfully read out: $R⋅q_0​$
  • Residual charge remaining in pixel: $(1−R)q_0​$

During the next exposure interval Δt, a small charge, $q_Δ$, is integrated.

  • Total charge before 2nd readout: $q(1)=(1−R)q_0​+q_Δ​$

After readout this will be reduced by (1-R) and so for our N-th readout we have,

  • Total Charge before Readout N: $q(N−1)=(1−R)^{N−1}q_0​+∑^{N−2}_{n=0​}(1−R)^nq_Δ​$

Since 1-R is less than 1, we have:

  • Readout at large N: $S(N)=R⋅(1−R)^{N−1}(q_0​−q_Δ​)+q_Δ​$

However, this is not quite sufficient. We need to account for charge trapping in the silicon. This is a slower process compared to the above. For purposes of numerics, we can approximate the effect as thermal-like decay in N.

$$S(N)=R(1−R)^{N−1}(q_0​−q_Δ​)+q_Δ​⋅e^{−γ(N−1)}$$

Let's see if the model works. We collected data sets as above using a green LED with our 16 bit sensor board and Teensy 4 controller, and varied the number of pulses to the shift gate, each pulse is 4V and 1usec. The following figure shows the residual carry-over versus the number of pulses to the shift gate for different values of the LED "on" time and different periods of the shift gate. We fit each of these to our model. We see that the curves are largely insensitive to the initial intensity and pulse period, R ranges from 0.44 to 0.54 and γ ranges from 0.082 to 0.10. For these values, the residual charge carry-over signal should fall below dark noise within 20 pulses of the shift gate.

Residual charge (ghosting) as a function of number shift pulses fit to "charge shovel" model with charge trapping.

But, does it work? The following shows a full scale exposure followed by 20 clearing pulses and then the "off" frame, collected as above. As predicted, the residual charge signal is indeed under the dark noise.

Residual charge clearance following a full scale, as above (a) LED on and (b) the LED "off" frame following the "on" frame and 20 clearing pulses, 4V, width 1usec, period 10usec.

We see that residual charge is a necessary part of the physics of the CCD detector device and features multiple regimes for shifting charge withing the device. We can mitigate the residual charge effect to prevent "ghosting" in our data by driving the shift gate with adequate voltage, current and time, combined with pulsing or idling the shift gate between exposures. Both idling in PLM mode and clearing in PIT mode are implemented in our firmware in a way that preserves deterministic timing for both timed and interrupt driven operation - as evidenced in the reported performance metrics.

Further details of CCDs and ADCs

In this section we revisit the CCD and ADC to fill in some of details and implement a few tricks that enable us to achieve the metrics that we listed at the top of the README. So far, we learned how to design an analog front end capable of linear low noise response in a mixed signal environment and interface it to an ADC, we learned how to drive the gates, and we learned how to clear residual charge. The next part involves the stop and go operation of the ADC and the interaction between its reference voltage system and the gate systems in the CCD. These effects are small thanks to our use of separate power busses and cuts in the ground plane. But to get to 0.2% INL requires one more step, or actually, two.

CCD readout

Let's see how the CCD readout works and what that means for our ADC. We saw in the preceding how charges move from the pixel to the analog shift register. Now we have to move charge along the shift register to the sensor output. We know from the datasheet that we need to drive 4 cycles in the master clock for each pixel we read from the sensor. So, this is the classic 4-phase potential well shift method. The following illustration shows how this works.

In (a) the upper part of the illustration, the shift register has a pattern of four electrodes per pixel, ϕ1​,ϕ2​,ϕ3 and ϕ4​. The pattern repeats for every pixel along the length of the CCD shift register. The trick is to form a well under several adjacent electrodes and advance the edges one at a time. We see this in (b), at t1 we have the well under three electrodes, at t2 the left edge moves, at t3 the right edge moves, and then repeat. The timing to accomplish this is shown in (c). This is sometimes described as a peristaltic pump for charge. Charge Transfer Efficiency (CTE) along the array is very high even after repeating this 3694 times to read the entire array.

4-Cycle CCD charge shift register; (a) electrode pattern, (b) "peristaltic" charge transfer and (c) clocking sequence. Homework #1: What is the next pattern at t4 in (b). Homework #2: Show how (c) gives rise to (b).

At end of the CCD charge shift register, we have to hold the charge for four clock cycles on a sample and hold capacitor and convert it to a voltage. The classic circuit for this is shown in the following [as described in the classic paper by J R Janesick et al, Opt Eng 26(8) pp 692-714 (1987)]. (Toshiba shows this as a black box with a 0.5K resistor in series with the output.) Negative charge carriers (blue) come from the end of the shift register on to the capacitor. An N-MOSFET follower then gives us a voltage at the output pin. The other MOSFET resets the capacitor to $V_{os}$ between pixels. The output waveform from the datasheet is shown in the lower right corner and we now understand why the output is a negative going voltage from a positive $V_{os}$.

Classic CCD sensor output stage, dual mosfet follow and reset.

For convenience, here is a sketch of the first stage in our analog front end circuit where we convert the sensor output to a differential pair. We use two OPAMPs (or a dual package) with one configured as a follower and the second as a shift and invert.

Now let's look at the timing for reading the CCD output. The following scope screen shows the master clock (CLK) on channel 1 (yellow), the integration clear gate (ICG) on channel 2 (blue), the positive copy of the CCD output signal on channel 3 (purple) and the conversion start (CNVST) signal to the ADC on channel 4 (green). This is a four channel scope so we don't have a lead for the shift gate. But, it is the combined SH & ICG that starts the readout. As described above, we have a new pixel at the leading edge of every fourth clock pulse. The CNVST is timed to start the SAR at just 100nsecs before the next clock at end of the pixel. We will see why this is important in the next section.

Scope trace, inverted-shifted sensor output purple, CNVST green, ICG blue, and master clock yellow. A new pixel arrives at every fourth clock. CNVST is timed to start the SAR just before the next clock.

ADC internals

Now, let's take a look at what is happening inside the ADC and learn why the CNVST timing that we just described in the preceding section is important. Here is a model for what is inside of a typical SAR type ADC. Whereas earlier we showed an equivalent circuit with a single sampling capacitor, the actual circuit for an n-bit SAR involves a bank of capacitors of values $C_0$ through $C_n = 2^{n-1} C_0$. The sampling switch is still with us, and labeled "S" in the diagram but it also involves switches on each of the capacitors.

SKetch of ADC internals with DAC and successive approximation engine.

Here is how it works.

  • Step 1: "S" is closed and all of the capacitors are switched onto $V_{in}$. The capacitors charge up to a total $Q = C_{total} \times V_{in}$. The top plate is at ground and so the charge is actually $-Q$.
  • Step 2: At the conversion start signal, "S" opens and the negative charge is trapped on the top plate.
  • Step 3: The SAR logic begins, the bottom plate switches are flipped one by one to $V_{ref}$. At each step the voltage on the top plate becomes $V_{+} = -V_in + V_{ref}\sum a_{m}/2^m$, where $a_m = 1,0$ depending on the switch position. The comparator detects this as greater than or less than 0, leaves the switch set or not accordingly, and moves to the next switch.

(The above is sufficient for the present discussion. As an exercise, the reader can try to invent a scheme to accomplish this with a single ended supply or a differential input.)

For a 4 bit ADC, the process might look something like this. Notice that the internal clock for the ADC SAR engine has to run at least as fast as the number of bits n times the sample rate.

SAR engine stepwise determination of a digital representation.
From https://en.wikipedia.org/wiki/File:Successive-approximation_ADC_example_working_value_1.svg
Public Domain via CC0 1.0.

Voltage reference stability: Virtual buffering

It is easy to see that the output from the SAR is going to depend on the $V_{ref}$, and that there can be a sizable kickback in the circuit that supplies $V_{ref}$ especially when the switch closes to connect $V_{ref}$ to thr first capacitor (the largest one in the array). Added to that, some ADCs go into standby or power down and then wake up on the first CNVST pulse. That is a less ideal situation for our case involving a CCD where we alternate between exposure and readout. Buffering the reference voltage would provide some mitigation for this. But a more complete solution - idling the CNVST between readouts - can act as a virtual buffer to provide electrical as well as thermal stability with the ADC in its ready to go state.

Proof is in the pudding. Here is the "before and after" implementing a CNVST idle engine. This is the ADC output converted to voltage scale at the sensor for easy comparison to our noise data. The x axis is time as pixel number $n \times 4 \times \text{clock period}$. Without idling the CNVST, we see ringing. When we idle the CNVST, the ringing disappears. We will come back to this when we present the timing engine.

Wakeup ringing in the ADC is resolved by idling the CNVST. (a) Ringing appears on exiting standby. (b) No ringing with the CNVST pin idling to maintain stability in the voltage reference system. Then CNVST is re-synchronized to the master clock during the ICG pulse.

CCD-ADC interaction: When to SAR and when not to SAR

In the above we see that moving charge along the CCD register to the output entails driving four sets of gates times 3694 pixels. The input is the master clock but this is a logic input, internally the device is driving four large capacitances. They are driven in pairs, ϕ1 goes high when ϕ3 goes low, and etc. But nonetheless and unavoidably some of this appears on the power and ground planes. And that is a path to effect the result produced by the SAR if it is in the midst of a conversion when the next shift gate happens. Again "proof is in the pudding".

The following shows the residuals from fitting a series of intensities and exposures times from a comparatively stable green LED, under two settings for the timing of the CNVST relative to the clock, -100nsec and -300nsec. The SAR runs until 700 nsecs following the CNVST pulse and we are running the CCD from a 600nsec clock.

In the first graph the CNVST pulse occurs at 100 nsecs prior to the next clock at the end of the pixel's 4 clock cycle. The last bits in the SAR conversion may overlap the second clock pulse, but the lower bits are also closer to the noise floor. So we refrain from drawing conclusions from the rise at low intensity.

In the second condition, the CNVST pulse is positioned 300 nsec prior to the clock. Here we do see a significant hump in the residuals and it is easy to imagine how it might reflect a disturbance from the CCD clock. We find that this is a repeatable result.

Residual error as a function of well usage, with CNVST at -100nsec and -300nsec relative to the pixel transition.

Orchestration and the "Idling Engine" State Machine

To achieve nanosecond-level determinism, eliminate the “ghosting” artifacts common in CCD systems, and achieve stable linear response in the complete path from sensor to ADC, this architecture—rather than relying on simple software-triggered loops—instead utilizes a Hardware-Locked State Machine based on the i.MX RT1062’s FlexPWM modules.

Deterministic Execution vs. Software Latency

In traditional microcontroller-based spectrometers, the timing of the conversion start (CNVST) and the CCD gate signals (SH, ICG) is often subject to interrupt latency or firmware "jitter." This results in inconsistent pixel-to-pixel integration times and baseline drift.

Our approach utilizes the "Idling Engine" concept: the timing engines run continuously. When a capture is initiated, rather than starting a "cold" set of clocks, the system transitions from a virtual buffer and high-speed charge-clearing state to a phase-locked readout sequence. This ensures:

  • Thermal and Electrical Equilibrium: The system remains in a constant state, avoiding the "first pixel" and "first frame" transients common in systems that start/stop clocks intermittently.

  • Phase-Locked Sampling: Sampling occurs with a constant phase relationship relative to the shift clock. This allows the signal to settle reliably to within 1 LSB while ensuring the ADC conversion avoids contamination by switching transients.

  • Precise Exposure Control: Exposure timing is hardware-locked and tightly controlled across the full dynamic range of the instrument.

Implementation

The system utilizes four submodules of a single FlexPWM module to implement the CLK, SH, ICG, and CNVST signals, with dedicated state engines managing the transitions. The following flowcharts document the initiation and hand-off logic for these engines.

Start logic supports command triggered initiation.

The shift gate ISR serves as the exposure and SH idle controller with hand-off to ICG to initiate readout.

The integration clear gate ISR occurs only at the end of an exposure, on the leading edge sets up the CNVST signal and readout and launches the readout on the trailing edge. ICG flow chart

The conversion start ISR fires for each pixel to assert the CNVST pin and command a 16 bit read over SPI, and implements the logic and actions for end of frame and end of frameset.

The resulting idle patterns are shown in the following. Note the resynchronization of the CNVST signal (green) prior to start of readout initiated from the ICG (blue) and the idle pattern in SH (purple) suspended during exposure and again during readout. The clock signal (yellow) runs on the scale of MHz while the display on the scale of 100KHz.

CNVST and SH idle signals maintained by the FlexPWM and state engines.


Appendix A - Quick command list

The following is a subset of the commands implemented in the TCD1304 firmware and in the Python user utility (indicated as CLI). Enter the command "help" for a more complete listing and consult the source code for further information.

Quick command reference
See help for more details.

Level Command / Sequence Technical Context
High-Level (Automated) read [n] [exp] PIT Mode: Executes n frames back-to-back with deterministic hardware timing. (2 arguments).
read [n] [exp] [int] Pulse Loop Mode (PLM): Executes n frames with a defined interval between starts. (3 arguments).
trigger [n] [exp] [int] Arms the system for a hardware trigger. Uses the same argument logic as read.
Intermediate (Orchestrated) setup pulse [clk sh offset icg offset] Explicitly configure individual pulse sequence.
setup pit exp [n [offset]]] Explicitly configures PIT registers for high-precision, back-to-back pulse sequences.
setup pulse loop [exp] [int] [n] [clk sh offset icg offset period] Explicitly configure pulse loop mode acquisition.
start / wait / stop Lifecycle control. Use wait for script synchronization in @filename execution.
Configuration set clock [seconds] Sets the master clock period (e.g., 0.0000004 for 2.5MHz operation).
configure timing [name val ...] Override timing parameters clk period, gate durations and offsets ... (see help)
configure clearing pulse [n [period]] Defines SH pulses between frames in PIT mode to flush residual charge and mitigate ghosting.
configure trigger Defines pin, edge (rising/falling/change), and pull-up/down logic.
System/Low level flexpwm [submod|mask] [ops] Lower level access to the flewpwm module (see help flexpwm)
pins Print configured digital I/O pin usage
set pin n val Set pin n high|low|output|pullup|pulldown|nopull|input
pulse pin n usecs Toggle pin n for usecs and toggle back
toggle pin n Toggle pin n
read pin n Read and report pin n as high or low
adcs n Read the first n analog inputs
version Show firmware version information
configuration Show hardware configuration, number of pixels, sensor type, etc
[store] coefficients [a0 a1 a2 a3] Store/report coefficients for wavelength, or other units
[store] units [string(<8chr)] Store/report units for the coefficients
[store] identifier [string(<64ch)] Store/report identifier string for this unit
temperature Report CPU temperature
reboot Commands a controller reboot
Python CLI add all [after n] Add data frames to a single frame (omitting first n frames)
add ensemble Add data frames collating by frame number
clear Clear acquired data frames and busy flag
save filespec Save acquired data frames to disk file
help [key] Display help text
@filespec [args] Execute commands from diskfile, args available as batchpars[]
!command Shell command
[lpar]=[rpar] Execute as a line of python code using exec() or with only '=' list namespace
for a in [0,1,2]: @testscript "%.2f"%(a_) Loop command
quit|exit|ctrl-c Quite and exit the Python TCD1304Controller program

About

TCD1304 device provides highly reproducible, linear response for precision photo-/radio-metric spectrometry and holographic imaging. Includes gerbers, BOM, firmware, library and user app.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Contributors

Languages