Keep experimental data in the lab
Measurements, unpublished sample details, and instrument logs can stay on the lab’s systems when inference and logging are configured locally. The lab controls access and decides what to keep.
UT Austin M.S. Computer Science thesis · Ongoing research
Small Language Models
for X-Ray Peak Centering
A small language model turns detector measurements into checked motor adjustments. This UT Austin thesis studies simulated X-ray peak centering, including detector noise, imperfect motor response, and an expanded three-motor interface. The aim is scientific control that laboratories can run locally.
Materials science
What does a Bragg peak tell us?
The atoms in a crystal form a repeating pattern. When X-rays scatter from those atoms, the waves add together at certain angles and produce a strong signal called a Bragg peak. On a detector, a reflection from a single crystal can appear as a bright spot.
Where the peaks appear tells scientists about the spacing and orientation of the crystal lattice. How bright they are carries information about the atoms and their arrangement. Together, these measurements help scientists work out a material’s structure and track how it changes during an experiment.
Background: X-ray diffraction at Diamond Light Source.
Why alignment matters
Before measuring a selected reflection, scientists need to align the instrument so they can record it reliably. That often means adjusting motors, checking the detector, and repeating. Automating this process could save time and let researchers spend more of the experiment on the material itself.
The task in this study
We focus on one part of alignment: bringing a selected spot to the center of a simulated detector. Computer vision tracks the spot, and the model proposes motor adjustments. The initial task used delta and chi; later local tests added phi using a supplied calibration profile. Centering prepares a reflection for measurement. Solving the crystal structure still requires more diffraction data and analysis.
Research context
Earlier work on AI-assisted X-ray experiments
Chen and colleagues demonstrated an AI system for X-ray experiments in simulation and at a synchrotron. This thesis focuses on training and evaluating a small model for one task within that kind of experiment: Bragg-spot centering.
We start with one task that we can train and test directly.
Using the same simulator and evaluation cases lets us compare the language model with a small numerical controller and examine where each succeeds or fails.
- Prior paper
- Broader agentic workflow with virtual and physical demonstrations.
- This thesis
- Simulated Bragg-spot centering, with an initial delta/chi task and a later local three-motor extension.
- Long-term direction
- Combine models trained for individual tasks into a larger experimental workflow.
Method
How the controller works
Computer vision measures the spot in each 128 × 128 detector image. The language model receives those measurements as text, along with calibration information.
Observe
Computer vision locates the selected spot and tracks its position after each move.
Propose
A fine-tuned Gemma model proposes a bounded adjustment using the available motors, calibration information, and recent feedback.
Check
Independent software checks the request. Accepted moves execute in the simulator, and the next measurement closes the loop.
- Simulated task
- Center a selected diffraction spot on a 128 × 128 detector in a simulated six-circle diffractometer
- Controller input
- Structured spot measurements, motor state, calibration information, and recent feedback
- Task-specific adaptation
- Trainable adapters on a pretrained Gemma model, with the base weights held fixed; training on TACC Vista
- Action checks
- Separate software validates the action format, available motors, and movement limits before execution
One recorded episode
Centering a spot in three adjustments
Simulation evidence
Testing imperfect sensing and motion
Later local tests kept Gemma’s learned weights fixed while varying detector conditions. Each detector condition below used the same 256 starting poses, with delta, chi, and phi available.
- Added detector noise
- 236/256 centered (92.2%)
Gaussian intensity noise applied before computer vision measured the spot. - Higher background
- 240/256 centered (93.8%)
Increased background signal around the diffraction spot. - Persistent pixel defects
- 237/256 centered (92.6%)
Fixed detector defects across successive observations. - What counts as centered?
- Within 0.5 pixel under an independent simulator-geometry check, with a budget of 20 executed backend actions. All planned episodes remain in the denominators.
When the motor does not move as requested
Separate studies tested reduced travel, deadband that suppresses small commands, finite motor resolution, and reversal play. These conditions probe how the controller responds when commanded and actual movement differ.
Motor stress tests were separate from the detector evaluation above. Severe deadbands could prevent centering; the detector success rates do not measure motor-fault robustness.
Extending the interface to a third motor
The same frozen Gemma checkpoint was evaluated with phi available alongside delta and chi, using a supplied local calibration profile and no additional weight training. This demonstrates reuse within an expanded local interface in simulation.
These are local simulation results with modeled perturbations, not measurements of physical-instrument reliability. The numerical MLP baseline was more accurate in the balanced comparison. Physical validation and transfer to other instruments remain future work.
Earlier development results and baseline comparisons
The figures below retain the earlier 940-case development study, with 43 initialization groups and three training seeds per trained controller. They use a different study design and endpoint from the later independent geometric evaluation above.
Figure A · clean condition
Every trained seed reached 897/940 by 20 actions.
Figure B · mixed-fault condition
The controllers differed more when given fewer actions.
Fine-tuning made the task work for Gemma. Before training, the model made no successful corrections from an off-center starting point. After supervised fine-tuning, it could center spots through repeated measurements and adjustments.
The MLP was a strong baseline. Its success rate was similar to trained Gemma within 20 actions, and higher at five actions under mixed faults. Similar rates here do not establish statistical equivalence.
RFT did not improve success within 20 actions. Additional reinforcement fine-tuning sometimes reduced success when fewer actions were allowed. The measured results did not show a clear Gemma advantage over the MLP.
Local AI for laboratories
Why run a model locally?
We want labs to be able to fine-tune a model for their equipment and run it themselves. With downloadable weights and suitable hardware, a small model can use the lab’s measurements and calibration instructions to propose actions that software checks before execution.
No hosted-model API key
Local inference uses saved model weights rather than a hosted model API. An air-gapped deployment would also need the model files, dependencies, logging, and updates prepared for offline operation. A complete air-gapped installation has not yet been validated.
Adapt a pretrained model
Task-specific adapter training keeps the pretrained base weights fixed. Labs can retain the version they evaluated and choose when to update it, subject to the model’s license. Training can take place on research compute before deployment on a suitable local system.
Avoid sending every request over the internet
A local model does not need to wait for a response from a cloud service at each step. That removes network travel time and dependence on the service’s uptime and rate limits. The model still needs time to compute, so actual response times have to be measured on the chosen hardware.
From research compute to lab deployment
The goal is a controller that a lab can operate on its own systems, with a fixed model version and inspectable action logs. Local deployment also makes the lab responsible for its hardware, software maintenance, and access controls.
Our training and evaluation ran on TACC Vista. Testing on lab hardware, measuring response times, and setting up an air-gapped system are possible next steps.
Further reading: Gemma fine-tuning and running Transformers offline.
Computing at TACC
Training and evaluation on Vista
TACC Vista supported simulation, data preparation, Gemma adapter fine-tuning, inference, and evaluation across controllers and test conditions. The pretrained base weights stayed fixed during adapter training.
This work used the Vista system at the Texas Advanced Computing Center.
Research team
People
David Katz
M.S. Computer Science researcher
Department of Computer Science
The University of Texas at Austin
The University of Texas at Austin
Elias Stengel-Eskin
Assistant Professor, Computer Science
Department of Computer Science
The University of Texas at Austin
The University of Texas at Austin
Zhantao Chen
Assistant Professor, Mechanical Engineering
Walker Department of Mechanical Engineering
Oden Institute for Computational Engineering and Sciences
Oden Institute for Computational Engineering and Sciences
Contact
David Katz
M.S. Computer Science · The University of Texas at Austin