개요
Marin은 기초 모델 연구 및 개발을 위한 연구 프로젝트, 소프트웨어 플랫폼, 커뮤니티입니다. 대규모 언어 모델(LLM) 훈련에 중점을 두며, 데이터 정리, 토크나이제이션, 사전 훈련, 후속 훈련, 평가를涵盖합니다. 개방형 개발을 강조하며, 음성-텍스트(音文), DNA, 단백질 모델에 활용된 바 있습니다.
README 미리보기
# Marin\n\n\n \n\n\n \n\n\n\n\n> "*I am not afraid of storms, for I am learning how to sail my ship.*"\n> – Louisa May Alcott\n\n[Marin](https://marin.community) is a research program, software platform, and community for the research and development of [foundation models](https://en.wikipedia.org/wiki/Foundation_model).\n\nMarin's concern is training large language models. This includes data curation, transformation, filtering, tokenization, pretraining, posttraining, and evaluation. Beyond the artifacts, software, and infrastructure, behind these models, Marin is committed to openly sharing *all* of the process knowledge required to build these models.\n\nMarin's core value is **[open development](https://openathena.ai/blog/open-development-of-frontier-ai/)**. We document our processes, experiments, and decisions as they happen. Every step, from raw data to the final model, is recorded. Failed experiments are part of that record.\n\nMarin has also been used for building [audio-text models](https://github.com/marin-community/marin/issues/1699), [DNA](https://github.com/Open-Athena/marin-dna), and [protein models](https://github.com/Open-Athena/MarinFold). We encourage this work through the use of Marin as a library, in [marin/experiments](https://github.com/marin-community/marin-experiments).\n\n## Current work\n\n### Frontier mixture-of-experts\n\nOur current focus is pretraining, from scratch, and posttraining a large (5e24 model-FLOPs, 500 billion+ total parameters) mixture-of-experts model to succeed on tasks of importance to scientists and researchers.\n\n### Scaling suite\n\nDelphi is Marin's open scaling suite scaling a LLM recipe from 3e18 to 1e23 FLOPs, inspired by Pythia. It has three parts: a scaling recipe that maps compute budgets to model configurations, a scaling suite trained from that recipe on the Google TPU Research Cloud, and a scaling law that uses the smaller Delphi models to predict the larger ones.\n\nWe released:\n\n- **Checkpoints**