AutoSpectral in Honeychrome

olivertburton

May 26

19 min read

Updated: 23 hours ago

I'm happy to announce that the AutoSpectral workflow for unmixing is being incorporated into the Honeychrome application. Honeychrome is an open-source, Python-based software that you can download and run, interacting with it like commercial software such as FlowJo or FCS Express. Unlike those, it's free to use.

Most of the development of Honeychrome has been done by Samson Rogers, but I've chipped in for the integration of AutoSpectral. Version 1.0.0 is now out and includes the whole AutoSpectral workflow. This should enable anyone to use the unmixing tools, without having to interact with a black box or enter code. I'll cover more about Honeychrome itself, and how you can contribute it you're a developer, in a separate article. We'll probably also need an article about the cool and very cheap open-source spectral cytometer that accompanies Honeychrome as part of the Cytkit project. Today, let's launch into how to use Honeychrome for unmixing of spectral flow cytometry data.

Note: Honeychrome now supports most major spectral cytometers. If you have data from an instrument that is not supported or does not work, reach out or post an issue on the GitHub.

Update: This article has been updated to changes in version 1.0.0.

Installation

First, download and install Honeychrome. Head over to Cytkit and download the installer for your operating system. This should be as simple as downloading and double-clicking to run the installer. If you're more computationally inclined, you can also clone the GitHub repository and run it directly from Python.

We're going to need some data. If you want to do unmixing, you need to start with files containing the raw data. On the ID7000 and the Aurora, you get separate raw files; on BD's machines, you get the raw and unmixed data all in the same file. I'm going to be using the data from the AutoSpectral workflow example, which you can download here, if you need something to work with.

Getting Started

Now, let's open Honeychrome and start working.

You should see this after clicking on Honeychrome's bee "sidekick" icon:

We're going to create a New Experiment.

Type in the name of your experiment. I'm going to call mine AutoSpectral Demo.

To start, you'll get a blank experiment, looking like this.

Head to the View menu up top and untick Acquisition Panel and Gains Panel to hide those and give yourself more space. Those panels are useful when you're using Honeychrome to acquire samples on a cytometer, but we don't need them right now.

Loading Data

Now, first thing is to add our data. Honeychrome organizes your data into folder in the Experiments folder, which will be created on the root of your user directory tree. The simplest way (for most users) to do this is to use the "Import FCS Files" option in the File menu:

Clicking that brings up this window:

If you use Option A, you bring up the folder for the Honeychrome experiment you're working on, and you can drag in FCS files or folders into the Raw folder.

Since you probably already have your FCS files organized nicely in a folder from the cytometer, you can use option B. For this, we're just going to tell Honeychrome where the data already are, no need to move or copy them.

The fields on the righthand side are selectable, so you can click on these to chang the location, telling Honeychrome where each item is. Let's change the location of the "Single Stain Controls Folder" to the copy of the data I've already got. Click on the "Raw/Single stain controls" field to change it.

Finding the existing single-stained controls folder

After linking:

And we can do the same for the "Raw Samples Folder":

Hit "Okay" once you've linked everything. Hit "Update Experiment Configuration".

Honeychrome has found the files and updated the experiment configuration on the basis that these files are from the 5-laser Cytek Aurora.

Et voila! Samples appear:

Notice that any subfolder organization of the "Raw Samples Folder" is maintained.

Unstained samples will be marked blue; bead samples will be marked green. Right-click on any sample to change these markings.

Interacting With Data

Let's take a moment to look at the data before we jump into the unmixing workflow. Click on any of the samples to load the data on the screen, which by default will be showing the Raw Data tab.

Here's my CD4 BV421 single-stained cell control:

We've got a default FSC/SSC gate and a singlet gate. Then we've got a spectral ribbon plot showing us the data in all the detectors. There's also a 1D histogram for each detector if you scroll down. Gates can be moved, or they can be adjusted by dragging on any of the corners with a small box icon. To change which gate is used to select the data displayed on a plot, click on the title. For example, we could change the spectral ribbon plot to display all events (root), if we wanted to:

The transformations on the axes are adjustable with the mouse, in a way that will likely be familiar to users of the ID7000. Let's create a new plot to play with. Either click the big + sign to create a new plot:

Or, right-click on an existing gate to create a new plot from that set of gated events:

By default, these new plots will appear at the bottom, after all the other plots, which maybe should be changed for the Raw Data tab. Anyway. Click on the axis label markers to change which parameters are displayed:

Drag on the numbers to change the transformation (scaling):

If you drag near zero, you adjust the compression (width basis) of the biexponential transform.

If you drag on the upper portion, you can change the range, increasing or decreasing it. Note: these are not intended to be examples of what the transformation parameters should be, merely examples of how to set them.

Click on another sample to load that data, and so on.

As of version 1.0.0, Honeychrome has per-sample gating. This means you can change a gate's position on a specific sample. To use that, right-click on a gate and select "Customise 'gate' for this sample".

Unmixing using the Spectral Process

The Spectral Process tab is where you isolate your spectral signatures for your fluorophores. Today we're doing unmixing, but you would also use this tab for performing compensation if you were doing conventional flow. There are two workflows for the isolation of the spectral profiles. The basic one uses a standard positive/negative approach, analogous to what you'd get on any cytometer, but largely automated, so it should be pretty quick. I'm not going to touch on that too much today, focusing instead on the AutoSpectral control cleaning workflow that has been integrated here.

When you first open the Spectral Process tab, it will be empty, like so:

There is a Show Help button at the top that you can click for general guidance on how to use it:

This may be hidden again when you're done with it.

To use the AutoSpectral control cleaning workflow, tick the box for "Show AutoSpectral Cleaning". This should bring up the help section for the AutoSpectral control cleaning workflow, which you can again hide when you're done with it.

Let's follow the instructions, shall we?

First, let's change the gates in the Raw Data tab to include most everything.

We just want to be sure that we're capturing the brightest events somewhere in those gates. Beyond that it doesn't matter much, and it will matter even less in the next version.

Next, we need to mark our unstained samples. Marking your unstained samples tells Honeychrome the sample doesn't have fluorophore signals in it. This means the control can be used for determination of the background for single-stained controls and/or autofluorescence extraction. Honeychrome automatically detects anything with the label "Unstained" and marks it as unstained. That's why we've got blue droplets next to the "Unstained (Cells)" sample and the entire folder that I created called "Unstained samples".

As of Version 1.0.0, Honeychrome also automatically detects bead samples based on how consistent they are on scatter. This is important because it allows us to associate unstained beads with single-stained beads, and unstained cells with single-stained cells. This has been tested with a wide variety of compensation beads on various cytometers, but it's new, so there may be situations where it makes a mistake.

Unstained samples are blue, bead samples are green

That BioLegend Positive Beads control is my unstained beads. Let's mark that accordingly. Right-click and select "Mark as unstained".

For best results in version 0.8.3 onwards, set a default unstained negative sample for the cell controls. Pick the one that is most relevant to any single-stained cell controls you have. For instance, if you have an experiment using splenocytes for your single-stained controls, use unstained spleen here, even if your fully stained samples are something else. This unstained negative is used to figure out which signals in your controls aren't the fluorophore signals. If you don't have any single-stained cell controls (i.e., all your controls are beads), it doesn't matter which unstained negative is selected here.

Select the most relevant negative control for cells in the drop-down

In the Spectral Process tab, click "Auto generate spectral controls". If you have a lot of controls, this might take a minute or two. As of version 1.0.0, there is an option you can tick to automatically use the AutoSpectral control cleaning process when using "Auto generate". This can be ticked on/off in each experiment using the tick box in the image above, or, generally for all new experiments, in Edit/App Settings:

We get a filled in table and a filled in plot (Profiles Viewer) showing the fluorophores in our controls. Honeychrome has automatically detected the dyes (Label column), provided those are in the FCS file names. If you have put your markers in the file names and Honeychrome recognizes them, those will be automatically added to Antigen. You may change both the Label and Antigen to suit your preferences. Fill in stuff that's missing, since those labels get written to any FCS files you create and they get put on the plots in the Unmixed Data tab.

Check the details in the table. Are the Particle Type assignments correct? Is the Major Channel (Peak Channel) correct? Change these if not.

You also get a condition number (what Cytek refers to, confusingly, as a Complexity Index). The condition number for this panel is 1.8 (low). You'll get a warning if your condition number is greater than the number of fluorophores in the panel. This doesn't necessarily mean something is wrong, but it is a reasonably achievable benchmark, and a condition number well above the number of fluorophores in the panel generally indicates that one or more of your controls are "junk". You'll also get warnings if your fluorophore profiles don't look like they should, based on cosine similarity to the reference spectra shipped with Honeychrome. This QC is pretty useful for the FACSDiscover and Aurora, much less so for the ID7000.

If you've marked your unstained samples and set your Particle Type correctly, you'll automatically get pairing in the "Unstained Negative" column. The negative can be configured individually for each control using the drop-down. You should see all suitable negatives (matching Particle Type, marked as unstained), including the option to use an internal negative.

If you want to see the impact of using separate unstained samples on the spectral profiles, just hit the "Recalculate" button.

Let's start the AutoSpectral control cleaning workflow. As of version 0.8.3, this is faster and simpler. For best results, set unstained negatives, and check that your Major Channel (peak channel) is set to a channel that gets strong signal from your fluorophore.

Now, scroll down to the AutoSpectral Control Cleaning box below the Profiles Viewer. If you've used "auto-cleaning", this will already be done.

Spectral Profiles here are without cleaning, with separate unstained negatives

Hit "Clean Controls". This should be pretty quick in version 0.8.3 onwards.

The fluorophore spectra shown in the Profiles Viewer are now the "cleaned" versions.

These are pretty easy controls, but we've still managed to remove some of the spurious low-level signals off the violet and blue lasers. For more on how this works, check out the article on Colibri Cytometry.

If you want, you can view the impact of the control cleaning on any control. In the Control Cleaning box, tick the options to show the various plots.

Purple events are the brightest 250 positive events, which are the ones being used for the calculation. This is CD45 BUV395 on murine splenocytes, so we didn't really have an internal negative to speak of. The blue events are the scatter-matched selected events from the separate unstained sample. You can adjust the position of the gates here for the standard positive/negative approach, but this really isn't important for the AutoSpectral workflow.

Scatter-matching allows us to find the most similar cells in the separate unstained sample, on the basis that those will be best matched for background, giving us the most accurate spectral determination. See Mario Roederer's article. The scatter-matching approach in version 0.8.3 onwards is based off of kNN matching (developed by Nathan Laniewski for AutoSpectral), and is really quite precise.

Scatter-matching for CD11b-BUV805. We're trying to match blue events in the negative (left panel) to purple events in the single-colour control (right panel)

In version 0.8.3, noise exclusion is done automatically and rapidly in the background using a fast process called cosine filtering. Basically, we look for the events in the single-colour control that are most unlike the background we see in the corresponding unstained sample. This works very well to identify the brightest events that aren't affected by autofluorescence contamination. This was also developed in concert with Nathan Laniewski. More on this in a separate post.

As part of the AutoSpectral control cleaning workflow, you'll get Cosine QC plots. These compare your extracted fluorophore profile to reference spectral traces for the same fluorophore. Don't expect these to match exactly--they're not the same antibody, same protocol or same exact machine. If you're working with data from the ID7000, these may be quite far off since the detector voltage setup on that machine is standardised quite differently to other systems. The shapes should be roughly the same, meaning you should get peaks in the same parts of the spectrum. If these don't look right, go up and try to change your Major Channel, or check what's going on with your control(s). You will only get Spectral QC plots for fluorophores where Honeychrome has a known reference for that fluorophore on that particular machine. So, if you're using a new fluorophore we don't have and want contribute, head over to the corresponding spreadsheet database linked in this post.

There's a bunch of other info in the Spectral Process tab. You get a cosine similarity matrix, a Hotspot Matrix and your unmixing matrix, all plotted as heatmaps for visualization. Below that, you have NxN plots and Spillover Fine Tuning sections that you can use to adjust compensation on top of the unmixing (I don't recommend this, but this sort of freehand fiddling is widely done).

Once you have extracted spectral profiles, the data can be viewed in the Unmixed Data tab. There is no separate unmixing step--they're already unmixed.

Unmixed Data for CD3 APC single-stained bead control

Using the AutoSpectral Tab

The AutoSpectral tab now contains a set of four tools you can use to deal with autofluorescence, noise, spectral variability, and small spectral unmixing errors.

Autofluorescence

Let's start with the autofluorescence handling. There is a help section, which you can show or hide.

Before we start using it, I'm going to make a point that you should not use the per-cell AF extraction in combination with manual extraction of single or multiple AF signatures. So, by default, unstained samples are not included in Honeychrome's spectral model, and you shouldn't see any in the Spectral Process table or the Profiles Viewer. If you do, remove those and recalculate the profiles before using AutoSpectral AF.

Now, select the unstained sample you want to use as a source of autofluorescence from the drop-down (all cell-based samples marked as unstained should appear here).

Click "Extract AF Profile".

Here's what the liver looks like:

If you have multiple sources of AF, like I have multiple unstained tissue samples in this experiment, extract the AF from each one. You can, if you want, save the profiles for later use as a CSV file. Click on each set of profiles below the "Stored AF Profiles" plot to view them.

You have two controls on this process: the "SOM grid size" and de-duplication.

The grid size determines the search space for discovering different AF profiles. A larger grid can be slightly better for discovering rare profiles (rare cell types) in complex samples, but takes slightly longer and isn't generally necessary. The approach here already includes a refinement step that looks for problematic rare populations that "stick out" when the unstained sample is unmixed.
Removing near-duplicates filters the set of AF profiles, merging ones that have a very high cosine similarity. This reduces the computational time for the unmixing at, generally, a very minor difference in unmixing quality. Turning deduplication off will be a bit slower and will give more compressed negative populations.

Now, let's move to the unmixing process. We first want to assign our AF profiles to the samples. There's a tool here to let you play around with this, testing the impact on the unmixing. Let's assign the profiles from the unstained liver to the fully stained liver, then see how it looks.

Click on the sample on the lefthand panel to load it. The unmixing will proceed automatically, but might take some seconds for large samples.

Mouse liver, without or with per-cell AF extraction

This is a panel deliberately designed to minimize the impact of autofluorescence on the fluorophore channels. That is, this is more or less a best-case scenario for this tissue.

As with the other plots, change the transformations, parameters or gates applied by clicking on the relevant portions of the plot. Try other samples, etc.

Mouse lung, without or with per-cell AF extraction

Once you have assigned AF profiles to a sample, per-cell AF extraction will be applied to the unmixing throughout, including in the Unmixed Data tab. Deselect the AF profiles to remove this and revert to standard unmixing without AF extraction.

As of version 1.0.0, there are more tools to help you assign the profiles to samples. If you've arranged samples into folders (groups), you can assign a profile to a given folder. You may also try the "Auto-assign from Scatter" button, which will match AF profiles to your samples based on how similar the distribution of events is on FSC/SSC. This has worked well in experiments such as the example here, where the scatter profiles are easily distinguishable by eye.

Noise Modelling

With flow cytometry data, we have multiple sources of noise that confound the accuracy of our measurements. For our purposes, we can think of there being primarily two types: a background hiss or hum in the detectors that limits the sensitivity, and a component that changes as we add more signals in the form of light. Both of those can vary by detector, and the second one will depend on what is happening in your panel on a per-event basis. For a cell with high expression of APC, there will be lots of red light emitted, which create more noise (uncertainty) in the detectors for red photons.

The noise modelling in the AutoSpectral tab takes the information from the unstained control and the single-stained controls to model how these signals affect the fully stained sample. This is a sort of weighting, like Poisson weighting, with an extra element of calculating how likely each event is to be positive for each fluorophore using the spillover spread model. The net result is that we can reduce the spread somewhat. The impact is generally minor or negligible in small panels from the Aurora and FACSDiscover, which have detectors with consistently low noise across the board, but increases as spillover spread increases and on the ID7000, where the detector noise appears to be quite variable.

With spectral cytometry data, we have lots of data, more measurements (detectors) than things we want to measure (fluorophores). When we use ordinary least squares, we treat all of these measurements as equally trustworthy. With weighting, we are effectively saying, some of this information is less reliable, and we should pay more attention to the part that is likely to be less noisy.

For the noise model, the best control is unstained beads. I recommend, for this and for unmixing generally, to only ever use BioLegend or Slingshot (SpectraComp) beads.

There is some help text, which may not be very helpful, but gives the theory.

All you need to do to use it is click the "Model Noise" button. The unstained cell and unstained bead controls (as available) should be automatically selected if you have marked them appropriately at the beginning.

Cells and beads were auto-selected, click Model Noise

Once we do that, we again get a comparison to allow us to evaluate whether we want to use this aspect in the unmixing. The noise modelling is slower to calculate because it requires actually processing each event's data individually. That said, it has been fast enough in testing that we have left it as an option for live unmixing in Honeychrome.

The impact of the noise weighting will depend on your panel and instrument. Two examples:

With only 7 colours on the Aurora, this makes no difference

A 40-colour panel on the ID7000 shows smaller negatives

The ID7000 comparison here is against standard weighted least squares with per-cell AF extraction, both of which already reduce the noise.

Once run, the noise model is automatically on and used throughout Honeychrome. Untick "Use the noise model..." to turn it off.

Spectral variation

Fluorophores, particularly tandem dyes, can vary in their spectral signatures. I suspect that for some dyes, there is not a single chemical species present, but rather a distribution. This is well known for PerCP-Cy5.5, which is only about 50% PerCP-Cy5.5, the other half being plain old PerCP. AutoSpectral includes an approach to try to deal with this variability by quantifying it and assigning cells to specific variants of the spectrum during the unmixing process. By allowing the spectrum to "flex" slightly, within the bounds measured from the single-stained control, we try to counteract some of the spread caused by variation. This approach is now in Honeychrome.

In order to implement this in Honeychrome for live unmixing, it is a light touch approach. Fewer variants are measured.

There is again a help section:

Click "Compute Spectral Variants".

You then get spectral profiles for each fluorophore with enough positive events to assess in the single-stained control. The blue line represents the main profile you measured in the Spectral Process tab. The fainter red lines are the variants found.

We see variability for PE-Cy in the blue detectors (cross excitation by the 488nm laser), and in YG1-YG3, which is the PE parent component. Some cells have a higher ratio of PE:PE-Cy7 on them.

We can evaluate the unmixing, with and without variants taken into account. Again, for this small panel, there is minimal impact.

For this 40-colour panel, compensation beads were used, creating an unmixing error between FITC and Spark Blue 550. Unmixing with spectral variants catches some of that.

The spectral variants and noise modelling are not currently compatible, so you'll want to use one or the other. I generally use the noise model.

Spectral Correction

This is new territory and a new technique, although Sony and Ozette both offer something along these lines, I believe. The Spectral Correction provides a statistical approach to detecting and correcting small unmixing errors due to mismatches between the single-stained controls and the fully stained samples. By this, I mean skewing (compensation-like) errors that can result from using compensation beads as controls, inadequate brightness in the controls or differential tandem breakdown between the control and the sample (differences in handling and light exposure are common problems). This is not perfect. It has, however, been tested on a dozen or so data sets spanning the Aurora, ID7000 and FACSDiscover, with generally decent results. If you observe bad results, first, don't use the result (see below), and second, if you are willing to share data and feedback, I'm happy to improve it.

I will write up the method in detail in a separate post, but there are two main ideas here:

Correlations in the data (for example, more PE signal as PE-Cy7 increases) are probably wrong. These may indicate errors.
In order for the error in a spectral profile to matter, the error has to contain signal that would otherwise go into a different dye. The error is "in-span", between markers in the unmixed space.

The second part is what makes this really hard to solve, at least as far as I can tell. At this point, the single-stained controls can't tell us much. Instead, we need some statistics to tell us where the error is likely to be coming from, how much we trust the measurement, and other factors such as physical-chemical properties of the dyes.

Anyway, using the method should be simple enough.

There is a help section, which may or may not clear things up (not sure I know how to explain this very well):

Start by selecting the sample(s) you want to use to find spectral unmixing errors:

You need to use fully stained samples for this. If there aren't enough positive events in a channel, there won't be enough evidence to correct any errors for that dye. This process is intended to be used to correct errors that are present globally, in all samples. If you have errors that are specific to a sample or that vary between samples, I would say you are dealing with sample preparation, staining, acquisition and/or autofluorescence issues and should not be using this method. An exception to that might be if you had a group of samples with a specific error not present in another group, for instance, tandem breakdown in tumour samples and not in peripheral blood. So, my recommendation is to either use all your fully stained samples (a selection of events will be taken), or use the ones you most trust to provide clean staining.

For this, I'm going to use the 40-colour ID7000 data, unmixed with compensation beads.

Once you select your samples, click "Fix Unmix":

While it is running, you'll get some status updates. This is not a fast process.

When it finishes, you can review the corrections side-by-side with the uncorrected data.

FITC is less skewed away from Spark Blue 550 after correction

Not perfect. Some room for improvement here. This is meant to be fairly conservative. It will struggle with situations where there is smooth coexpression between similar dyes, which is something you want to avoid in designing your flow panel anyway. For instance, in the example below, I've used IRF4 on Pacific Blue, with B220 for B cells on StarBright Violet 440.

Over-correction between similar dyes when co-expressed

These dyes are stable and not prone to unmixing issues, so the original pattern on the left is probably correct. B220+ B cells could be expressing slightly more IRF4 than other cells (naive T and NK would be the bulk of the B220- population). At the moment, the method calls this an potential unmixing error and corrects it, pulling the B cells back to be even with the rest. This particular 40-colour Aurora experiment was designed to contain difficult combinations, not to be a good panel.

This one's subtle, and I wouldn't have caught this. PE-Fire 700 is leaning in towards BB700 on the left.

Example from a 40-colour mouse Aurora panel

You get a table telling you which dyes have been "corrected", and a profile viewer showing the spectrum before and after.

You can also view them all at once using the heatmap:

Any dye whose "correction" seems wrong to you can be unticked, reverting to the original spectrum you obtained from the Spectral Process. For instance, we could untick that StarBright Violet 440 correction in the example with B220 vs IRF4. By default, once run, spectral corrections are applied to the unmixing throughout Honeychrome. You can turn this off by unticking this box:

Or change which samples receive the corrections:

You can also tell it which dyes to exclude from the correction process. So, if you have some idea already where the problem is, you can just run those dyes through correction. By default, everything is active and checked for problems:

Exporting the Data

Honeychrome offers advanced data analysis options, so you may wish to continue with the analysis in Honeychrome. If, however, you prefer to use workflows you're comfortable with, you'll be looking to export your unmixed data as FCS files for use elsewhere. Here's how you do that.

In the File menu, select "Batch Export FCS Files".

A pop-up will appear. Select the samples you want to export as unmixed FCS files. They will be exported with the labels you have added to the table in the Spectral Process tab, using any assigned AF profiles, and including any spillover fine tuning (compensation adjustment) you have applied.

Hit "Proceed" and the export will start.

All done! There is no "Save" button because the experiment is always saved. If you close it at any point, it will reopen to the point you left off at.