Summer Internship / Bloomberg Machine Learning Product and Research

Language modeling has been around for decades, but it has only been within the past few years that large (causal) language models have demonstrated generalization abilities that enable direct application to core problems in natural language processing.

LLM performance on novel tasks is frequently surprising, and outstrips the theoretical understanding of these models. Even basic questions such as how the models encode information and how they retrieve it are still being worked out. As optimization problems, these models are very complicated, both non-convex and overparameterized, and yet simple SGD solutions often are able to find good optima. Correspondingly, just as these models have been exhaustively tested on natural language, they also appear to work in adjacent domains like programming languages, and multi-modal contexts, but the full extent of their utility is still unknown. Furthermore, there are significant barriers to application of these models in production – the GPU and RAM required to perform inference with the largest of these models often prohibits immediate deployment. Finally, while LLMs are able to string together long coherent sentences, they have gaps in their ability to plan, to reliably produce intermediate steps towards a specific goal, and to use long sequences as context.

This summer, the CTO Machine Learning Product and Research group is looking for PhD research interns curious to explore the theory, engineering, and applications of large causal language models with the goal of producing significant research impact.

Some particular areas of interest:

Multimodal (structured and tabular data with language).
Code generation & code understanding.
Localizing knowledge w/in LLM, editing, and updating.
Integrating LLMs & external APIs.
Fine-tuning models for downstream tasks.
Methods for efficiently running the models.
Training with differential privacy.

Please send email to gmann16@bloomberg.net directly if interested.