Deep neural networks are becoming increasingly popular in more and more industries, and while recent developments in networks designed to play Go and online video games have excited the attention of the media, the use of these networks in research is also gaining traction.

Most recently, a group of researchers from Drexel University have built a neural network designed to predict the risk of infection with severe coronavirus disease 2019 (COVID-19) from the sequence of the spike protein. This study is currently available on the preprint server In Review*.

The Study

The researchers aimed to translate sequence data to viral phenotype using deep neural network models. The model can be trained to predict the function of a sequence of amino acids that are encoded with words in a sentence in language processing models, and then assigning them integer tokens.

Transformer architecture was used for sequence encoding. The Transformer is a modular multi-head structure, with each head composed of an attention layer and feed-forward neural network, with head outputs added and normalized to provide a sequence encoding. A convolutional neural network (CNN) is used with a kernel width of 1 – this helps to reduce the size of the transformer heads and keep computation efficient.

Neural networks often provide accurate predictions – but it can be difficult to explain how these predictions were created. The scientists added a forward attention layer and an intermediate NH-dimension connected layer to help classify, visualize and interpret the predictions.

The model was trained on a fraction of GISAID sequence data that contained metadata on the patient status – around 147,000 samples. Once all samples that could not be assigned to mild or severe were removed, ~54,000 samples remained. In order to create an accurate model, the researchers were forced to include demographic data, including age and gender, as these can significantly alter the predicted outcome of COVID-19 infection.

No consistent trends were found with regards to the extremely young/old, and the scientists worried that sampling bias could skew this data – with relatively few samples from the young and old, it could be more likely that only hospitalized individuals were sampled.

A consistent trend in frequency of severe outcomes of infection was found to relate to the sample collection date, with a sharp decrease since February 2021. This trend continues across time even as new variants emerge and rise to dominance – contradicting studies showing that the Alpha variant results in increased hospitalizations and ICU admissions. This reduction in severity is likely due to changes in the treatment of infected individuals and increased prevention tactics, as well as the emergence of monoclonal antibody treatments, as well as the beginning of mass vaccination programmes.

The researchers also found that the sequence data was affected by the area the sequences were obtained – with sequences obtained earlier in the pandemic more likely to be classified as severe, despite no increase in symptoms.

The deep learning model was able to predict a significant proportion of m8ild and severe classes, and when sequences and demographic data were both included in the model, predictions were significantly improved. When the deep learning model was benchmarked with a random forests (RF) algorithm, the deep learning model was found to make errors at a similar rate – which is impressive for a newly developed model. Unfortunately, the model could not detect a significant difference between rates of severity between genders – despite multiple studies showing more severe disease in men.

The researchers attempted to use the model to predict the severity of the Omicron variant. After controlling for age and date, the trained model was run with the same age date and gender inputs as predictions for the other variants in order to compare their severity. The model predicted significantly lower severity than Delta, a finding which is supported by the observations of multiple healthcare workers and researchers.

The sites with the greatest attention difference – implying particular relevance in predicting severity – included 69-70, an Omicron deletion, 95, where the Delta mutation has the T951 mutation, as well as several other sites of key mutations found in the more severe variants of concern

Conclusion

The authors highlight that they have successfully created a model that can make accurate and validated predictions for the Omicron variant, despite the fact that Omicron is significantly different from most previous variants.

They point out that this is a strong argument for the use of the model in predicting the behavior of future variants and could help to inform healthcare workers and public health policymakers of the impact of any future variants that emerge.

*Important notice

This study is a preliminary scientific report that has not yet been peer-reviewed and, therefore, should not be regarded as conclusive, guide clinical practice/health-related behavior, or treated as established information.

Written by

Sam Hancock

Sam completed his MSci in Genetics at the University of Nottingham in 2019, fuelled initially by an interest in genetic ageing. As part of his degree, he also investigated the role of rnh genes in originless replication in archaea.