Towards deployment-centric multimodal AI beyond vision and language

Xianyuan Liu; Jiayang Zhang; Shuo Zhou; Thijs L. van der Plas; Avish Vijayaraghavan; Anastasiia Grishina; Mengdie Zhuang; Daniel Schofield; Christopher Tomlinson; Yuhan Wang; Ruizhe Li; Louisa van Zeeland; Sina Tabakhi; Cyndie Demeocq; Xiang Li; Arunav Das; Orlando Timmerman; Thomas Baldwin-McDonald; Jinge Wu; Peizhen Bai; Zahraa Al Sahili; Omnia Alwazzan; Thao N. Do; Mohammod N. I. Suvon; Angeline Wang; Lucia Cipolina-Kun; Luigi A. Moretti; Lucas Farndale; Nitisha Jain; Natalia Efremova; Yan Ge; Marta Varela; Hak-Keung Lam; Oya Celiktutan; Ben R. Evans; Alejandro Coca-Castro; Honghan Wu; Zahraa S. Abdallah; Chen Chen; Valentin Danchev; Nataliya Tkachenko; Lei Lu; Tingting Zhu; Gregory G. Slabaugh; Roger K. Moore; William K. Cheung; Peter H. Charlton; Haiping Lu

doi:10.1038/s42256-025-01116-5

Towards deployment-centric multimodal AI beyond vision and language

Xianyuan Liu, Jiayang Zhang, Shuo Zhou, Thijs L. van der Plas, Avish Vijayaraghavan, Anastasiia Grishina, Mengdie Zhuang, Daniel Schofield, Christopher Tomlinson, Yuhan Wang, Ruizhe Li, Louisa van Zeeland, Sina Tabakhi, Cyndie Demeocq, Xiang Li, Arunav Das, Orlando Timmerman, Thomas Baldwin-McDonald, Jinge Wu, Peizhen BaiZahraa Al Sahili, Omnia Alwazzan, Thao N. Do, Mohammod N. I. Suvon, Angeline Wang, Lucia Cipolina-Kun, Luigi A. Moretti, Lucas Farndale, Nitisha Jain, Natalia Efremova, Yan Ge, Marta Varela, Hak-Keung Lam, Oya Celiktutan, Ben R. Evans, Alejandro Coca-Castro, Honghan Wu, Zahraa S. Abdallah, Chen Chen, Valentin Danchev, Nataliya Tkachenko, Lei Lu, Tingting Zhu, Gregory G. Slabaugh, Roger K. Moore, William K. Cheung, Peter H. Charlton, Haiping Lu

Research output: Contribution to journal › Review article › peer-review

1 Citation (SciVal)

1 Downloads (Pure)

Abstract

Multimodal artificial intelligence (AI) integrates diverse types of data via machine learning to improve understanding, prediction and decision-making across disciplines such as healthcare, science and engineering. However, most multimodal AI advances focus on models for vision and language data, and their deployability remains a key challenge. We advocate a deployment-centric workflow that incorporates deployment constraints early on to reduce the likelihood of undeployable solutions, complementing data-centric and model-centric approaches. We also emphasize deeper integration across multiple levels of multimodality through stakeholder engagement and interdisciplinary collaboration to broaden the research scope beyond vision and language. To facilitate this approach, we identify common multimodal-AI-specific challenges shared across disciplines and examine three real-world use cases: pandemic response, self-driving car design and climate change adaptation, drawing expertise from healthcare, social science, engineering, science, sustainability and finance. By fostering interdisciplinary dialogue and open research practices, our community can accelerate deployment-centric development for broad societal impact.

Original language	English
Pages (from-to)	1612-1624
Number of pages	13
Journal	Nature Machine Intelligence
Volume	7
Issue number	10
Early online date	21 Oct 2025
DOIs	https://doi.org/10.1038/s42256-025-01116-5
Publication status	Published - 31 Oct 2025

Data Availability Statement

Source data are provided with this paper. They are also available at
https://github.com/multimodalAI/multimodal-ai-landscape, where
they will be updated annually.

Acknowledgements

This work was enabled and supported by the Alan Turing Institute. We thank T. Chakraborty and C. Li for inspiring this work, D. A. Clifton for his support and T. Dunstan for contributing to the climate change adaptation section. The views expressed in this material are those of the authors and do not necessarily represent the views of their affiliated institutions or funders.

Funding

J.Z. is supported by donations from D. Naik and S. Naik. S.Z. is supported by EPSRC (grant EP/Y017544/1). T.L.v.d.P. was supported by EPSRC (grant EP/Y028880/1). A.V. is supported by UKRI CDT in AI for Healthcare (grant EP/S023283/1). A.G. is supported by the Research Council of Norway (secureIT project 288787). M.Z. is supported by EPSRC (grant EP/X031276/1). C.T. is supported by UKRI CDT in AI-enabled Healthcare (grant EP/S021612/1). R.L. is supported by the Royal Society (grant IEC\NSFC\233558). L.v.Z. is supported by NERC (grant NE/W004747/1). O.T. is supported by UKRI CDT in Application of Artificial Intelligence to the study of Environmental Risks (grant EP/S022961/1). Z.A.S. is supported by Google DeepMind. O.A. is supported by NIHR Barts BRC (grant NIHR203330). T.N.D. is supported by UKRI CDT in Accountable, Responsible and Transparent AI (grant EP/S023437/1). L.F. is supported by MRC (grant MR/W006804/1). N.J. is supported by the EU’s co-funded HE project MuseIT (grant 101061441). M.V. is supported by St George’s Hospital Charity. A.C.-C. is supported by EPSRC (grant EP/Y028880/1). H.W. is supported by MRC (grant MR/X030075/1). C.C. is supported by the Royal Society (grant GS\R2\242355). T.Z. was supported by the Royal Academy of Engineering (grant RF\201819\18\109). G.G.S. is supported by EPSRC (grant EP/Y009800/1). P.H.C. is supported by BHF (grant FS/20/20/34626). H.L. is supported by EPSRC (grant UKRI396).

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1038/s42256-025-01116-5

2504.03603v2Accepted author manuscript, 1.08 MBLicence: CC BY

https://www.nature.com/articles/s42256-025-01116-5

Cite this

Liu, X., Zhang, J., Zhou, S., van der Plas, T. L., Vijayaraghavan, A., Grishina, A., Zhuang, M., Schofield, D., Tomlinson, C., Wang, Y., Li, R., van Zeeland, L., Tabakhi, S., Demeocq, C., Li, X., Das, A., Timmerman, O., Baldwin-McDonald, T., Wu, J., ... Lu, H. (2025). Towards deployment-centric multimodal AI beyond vision and language. Nature Machine Intelligence, 7(10), 1612-1624. https://doi.org/10.1038/s42256-025-01116-5

Liu, X, Zhang, J, Zhou, S, van der Plas, TL, Vijayaraghavan, A, Grishina, A, Zhuang, M, Schofield, D, Tomlinson, C, Wang, Y, Li, R, van Zeeland, L, Tabakhi, S, Demeocq, C, Li, X, Das, A, Timmerman, O, Baldwin-McDonald, T, Wu, J, Bai, P, Al Sahili, Z, Alwazzan, O, Do, TN, Suvon, MNI, Wang, A, Cipolina-Kun, L, Moretti, LA, Farndale, L, Jain, N, Efremova, N, Ge, Y, Varela, M, Lam, H-K, Celiktutan, O, Evans, BR, Coca-Castro, A, Wu, H, Abdallah, ZS, Chen, C, Danchev, V, Tkachenko, N, Lu, L, Zhu, T, Slabaugh, GG, Moore, RK, Cheung, WK, Charlton, PH & Lu, H 2025, 'Towards deployment-centric multimodal AI beyond vision and language', Nature Machine Intelligence, vol. 7, no. 10, pp. 1612-1624. https://doi.org/10.1038/s42256-025-01116-5

@article{95b14c36bf9a48baa62d650e470abde1,

title = "Towards deployment-centric multimodal AI beyond vision and language",

abstract = "Multimodal artificial intelligence (AI) integrates diverse types of data via machine learning to improve understanding, prediction and decision-making across disciplines such as healthcare, science and engineering. However, most multimodal AI advances focus on models for vision and language data, and their deployability remains a key challenge. We advocate a deployment-centric workflow that incorporates deployment constraints early on to reduce the likelihood of undeployable solutions, complementing data-centric and model-centric approaches. We also emphasize deeper integration across multiple levels of multimodality through stakeholder engagement and interdisciplinary collaboration to broaden the research scope beyond vision and language. To facilitate this approach, we identify common multimodal-AI-specific challenges shared across disciplines and examine three real-world use cases: pandemic response, self-driving car design and climate change adaptation, drawing expertise from healthcare, social science, engineering, science, sustainability and finance. By fostering interdisciplinary dialogue and open research practices, our community can accelerate deployment-centric development for broad societal impact.",

author = "Xianyuan Liu and Jiayang Zhang and Shuo Zhou and \{van der Plas\}, \{Thijs L.\} and Avish Vijayaraghavan and Anastasiia Grishina and Mengdie Zhuang and Daniel Schofield and Christopher Tomlinson and Yuhan Wang and Ruizhe Li and \{van Zeeland\}, Louisa and Sina Tabakhi and Cyndie Demeocq and Xiang Li and Arunav Das and Orlando Timmerman and Thomas Baldwin-McDonald and Jinge Wu and Peizhen Bai and \{Al Sahili\}, Zahraa and Omnia Alwazzan and Do, \{Thao N.\} and Suvon, \{Mohammod N. I.\} and Angeline Wang and Lucia Cipolina-Kun and Moretti, \{Luigi A.\} and Lucas Farndale and Nitisha Jain and Natalia Efremova and Yan Ge and Marta Varela and Hak-Keung Lam and Oya Celiktutan and Evans, \{Ben R.\} and Alejandro Coca-Castro and Honghan Wu and Abdallah, \{Zahraa S.\} and Chen Chen and Valentin Danchev and Nataliya Tkachenko and Lei Lu and Tingting Zhu and Slabaugh, \{Gregory G.\} and Moore, \{Roger K.\} and Cheung, \{William K.\} and Charlton, \{Peter H.\} and Haiping Lu",

year = "2025",

month = oct,

day = "31",

doi = "10.1038/s42256-025-01116-5",

language = "English",

volume = "7",

pages = "1612--1624",

journal = "Nature Machine Intelligence",

issn = "2522-5839",

publisher = "Springer International Publishing",

number = "10",

}

TY - JOUR

T1 - Towards deployment-centric multimodal AI beyond vision and language

AU - Liu, Xianyuan

AU - Zhang, Jiayang

AU - Zhou, Shuo

AU - van der Plas, Thijs L.

AU - Vijayaraghavan, Avish

AU - Grishina, Anastasiia

AU - Zhuang, Mengdie

AU - Schofield, Daniel

AU - Tomlinson, Christopher

AU - Wang, Yuhan

AU - Li, Ruizhe

AU - van Zeeland, Louisa

AU - Tabakhi, Sina

AU - Demeocq, Cyndie

AU - Li, Xiang

AU - Das, Arunav

AU - Timmerman, Orlando

AU - Baldwin-McDonald, Thomas

AU - Wu, Jinge

AU - Bai, Peizhen

AU - Al Sahili, Zahraa

AU - Alwazzan, Omnia

AU - Do, Thao N.

AU - Suvon, Mohammod N. I.

AU - Wang, Angeline

AU - Cipolina-Kun, Lucia

AU - Moretti, Luigi A.

AU - Farndale, Lucas

AU - Jain, Nitisha

AU - Efremova, Natalia

AU - Ge, Yan

AU - Varela, Marta

AU - Lam, Hak-Keung

AU - Celiktutan, Oya

AU - Evans, Ben R.

AU - Coca-Castro, Alejandro

AU - Wu, Honghan

AU - Abdallah, Zahraa S.

AU - Chen, Chen

AU - Danchev, Valentin

AU - Tkachenko, Nataliya

AU - Lu, Lei

AU - Zhu, Tingting

AU - Slabaugh, Gregory G.

AU - Moore, Roger K.

AU - Cheung, William K.

AU - Charlton, Peter H.

AU - Lu, Haiping

PY - 2025/10/31

Y1 - 2025/10/31

N2 - Multimodal artificial intelligence (AI) integrates diverse types of data via machine learning to improve understanding, prediction and decision-making across disciplines such as healthcare, science and engineering. However, most multimodal AI advances focus on models for vision and language data, and their deployability remains a key challenge. We advocate a deployment-centric workflow that incorporates deployment constraints early on to reduce the likelihood of undeployable solutions, complementing data-centric and model-centric approaches. We also emphasize deeper integration across multiple levels of multimodality through stakeholder engagement and interdisciplinary collaboration to broaden the research scope beyond vision and language. To facilitate this approach, we identify common multimodal-AI-specific challenges shared across disciplines and examine three real-world use cases: pandemic response, self-driving car design and climate change adaptation, drawing expertise from healthcare, social science, engineering, science, sustainability and finance. By fostering interdisciplinary dialogue and open research practices, our community can accelerate deployment-centric development for broad societal impact.

AB - Multimodal artificial intelligence (AI) integrates diverse types of data via machine learning to improve understanding, prediction and decision-making across disciplines such as healthcare, science and engineering. However, most multimodal AI advances focus on models for vision and language data, and their deployability remains a key challenge. We advocate a deployment-centric workflow that incorporates deployment constraints early on to reduce the likelihood of undeployable solutions, complementing data-centric and model-centric approaches. We also emphasize deeper integration across multiple levels of multimodality through stakeholder engagement and interdisciplinary collaboration to broaden the research scope beyond vision and language. To facilitate this approach, we identify common multimodal-AI-specific challenges shared across disciplines and examine three real-world use cases: pandemic response, self-driving car design and climate change adaptation, drawing expertise from healthcare, social science, engineering, science, sustainability and finance. By fostering interdisciplinary dialogue and open research practices, our community can accelerate deployment-centric development for broad societal impact.

UR - https://www.scopus.com/pages/publications/105019488982

UR - https://arxiv.org/pdf/2504.03603

U2 - 10.1038/s42256-025-01116-5

DO - 10.1038/s42256-025-01116-5

M3 - Review article

SN - 2522-5839

VL - 7

SP - 1612

EP - 1624

JO - Nature Machine Intelligence

JF - Nature Machine Intelligence

IS - 10

ER -

Towards deployment-centric multimodal AI beyond vision and language

Abstract

Data Availability Statement

Acknowledgements

Funding

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this