概要
Marin は、基盤モデルの研究と開発のための研究プロジェクト、ソフトウェアプラットフォーム、およびコミュニティです。大規模言語モデルのトレーニングに特化しており、データキュレーション、トークン化、事前学習、事後学習、評価を網羅しています。オープンな開発を重視しており、音声テキスト、DNA、タンパク質モデルにもすでに使用されています。
README プレビュー
# Marin\n\n\n \n\n\n \n\n\n\n\n> "*I am not afraid of storms, for I am learning how to sail my ship.*"\n> – Louisa May Alcott\n\n[Marin](https://marin.community) is a research program, software platform, and community for the research and development of [foundation models](https://en.wikipedia.org/wiki/Foundation_model).\n\nMarin's concern is training large language models. This includes data curation, transformation, filtering, tokenization, pretraining, posttraining, and evaluation. Beyond the artifacts, software, and infrastructure, behind these models, Marin is committed to openly sharing *all* of the process knowledge required to build these models.\n\nMarin's core value is **[open development](https://openathena.ai/blog/open-development-of-frontier-ai/)**. We document our processes, experiments, and decisions as they happen. Every step, from raw data to the final model, is recorded. Failed experiments are part of that record.\n\nMarin has also been used for building [audio-text models](https://github.com/marin-community/marin/issues/1699), [DNA](https://github.com/Open-Athena/marin-dna), and [protein models](https://github.com/Open-Athena/MarinFold). We encourage this work through the use of Marin as a library, in [marin/experiments](https://github.com/marin-community/marin-experiments).\n\n## Current work\n\n### Frontier mixture-of-experts\n\nOur current focus is pretraining, from scratch, and posttraining a large (5e24 model-FLOPs, 500 billion+ total parameters) mixture-of-experts model to succeed on tasks of importance to scientists and researchers.\n\n### Scaling suite\n\nDelphi is Marin's open scaling suite scaling a LLM recipe from 3e18 to 1e23 FLOPs, inspired by Pythia. It has three parts: a scaling recipe that maps compute budgets to model configurations, a scaling suite trained from that recipe on the Google TPU Research Cloud, and a scaling law that uses the smaller Delphi models to predict the larger ones.\n\nWe released:\n\n- **Checkpoints**