项目简介
Marin 是一个用于基础模型研究与开发的研究项目、软件平台和社区。专注于训练大型语言模型，涵盖数据整理、分词、预训练、后训练和评估。强调开放开发，并已用于音文、DNA 和蛋白质模型。
README 预览
# Marin\n\n\n \n\n\n \n\n\n\n\n> "*I am not afraid of storms, for I am learning how to sail my ship.*"\n> – Louisa May Alcott\n\n[Marin](https://marin.community) is a research program, software platform, and community for the research and development of [foundation models](https://en.wikipedia.org/wiki/Foundation_model).\n\nMarin's concern is training large language models. This includes data curation, transformation, filtering, tokenization, pretraining, posttraining, and evaluation. Beyond the artifacts, software, and infrastructure, behind these models, Marin is committed to openly sharing *all* of the process knowledge required to build these models.\n\nMarin's core value is **[open development](https://openathena.ai/blog/open-development-of-frontier-ai/)**. We document our processes, experiments, and decisions as they happen. Every step, from raw data to the final model, is recorded. Failed experiments are part of that record.\n\nMarin has also been used for building [audio-text models](https://github.com/marin-community/marin/issues/1699), [DNA](https://github.com/Open-Athena/marin-dna), and [protein models](https://github.com/Open-Athena/MarinFold). We encourage this work through the use of Marin as a library, in [marin/experiments](https://github.com/marin-community/marin-experiments).\n\n## Current work\n\n### Frontier mixture-of-experts\n\nOur current focus is pretraining, from scratch, and posttraining a large (5e24 model-FLOPs, 500 billion+ total parameters) mixture-of-experts model to succeed on tasks of importance to scientists and researchers.\n\n### Scaling suite\n\nDelphi is Marin's open scaling suite scaling a LLM recipe from 3e18 to 1e23 FLOPs, inspired by Pythia. It has three parts: a scaling recipe that maps compute budgets to model configurations, a scaling suite trained from that recipe on the Google TPU Research Cloud, and a scaling law that uses the smaller Delphi models to predict the larger ones.\n\nWe released:\n\n- **Checkpoints**