The impact of AI and Language Models

Girton College's Supernumerary Fellow, Professor Ted Briscoe and PhD Student, Austin Tripp, presented their pioneering AI research into Large Language Models and using AI to design molecules at our recent Fellows' Research Evening. Discover more about what their talks focused on and their impact below. 

Professor Ted Briscoe: "Large Language Models (like ChatGPT): The Hype and the Reality"

Professor Briscoe Ted's talk focused on how ChatGPT has exposed an unprecedented number of people to cutting-edge natural language processing using large language models. It has also ignited a vigorous and often overblown public debate over the potential benefits, risks and capabilities of Generative AI. In the talk he explained the differences between 'small' and large language models, and showed via examples that, despite their impressive fluency and some 'emergent' capabilities like translation and question answering, they do not as yet fully learn the mapping between form and meaning encoded in the grammar of individual languages, often struggle to resolve pronoun references, and fail to infer the discourse relations between sentences. As such, they represent an impressive and useful step change in language processing capabilities if used with care, but artificial general intelligence remains a challenging and elusive goal that will likely require a significantly different type of model.

Ted has worked on statistical and robust parsing algorithms, computational approaches to lexicon acquisition and to representation of lexical, syntactic and semantic knowledge, textual information extraction from scientific articles and regulatory documents, models of human language learning and processing, and evolutionary models of language development and change. His recent work has mostly focussed on NLP and ML techniques in support of language learning.

Austin Tripp, PhD Student in Machine Learning: 'Using AI to design molecules.'

Austin gave a whirlwind tour of how AI can assist in drug discovery, including some of his own work in molecule generation and chemical synthesis planning. The focus of the talk was whether AI can be used to design life-saving medicines, and although there has been a lot of progress in this area, limited data and the multi-stage nature of drug discovery limit the potential of AI to "revolutionise" the field. Austin ended the talk with some thoughts on how recent advances in language models are likely to impact the field.

Austin's main research interest is in applied maths and the prospect of custom AI systems that could help increase the rate of scientific research. His PhD group specialises in Bayesian machine learning and some of the specific problems he is working on are:

• Applying Bayesian optimization to drug screening
• Molecular property prediction with quantified uncertainty
• Retrosynthetic planning (how to synthesize novel molecules)
• Meaningful evaluation of ML algorithms in chemistry (average loss on a test set can be misleading)

^{Image: Retro-fallback: retrosynthetic planning in an uncertain world}