Packt Publishing, 2023. — 258 p. — ISBN: 978-1804618257.
Foundation models have forever changed machine learning. From BERT to ChatGPT, CLIP to Stable Diffusion, when billions of parameters are combined with large datasets and hundreds to thousands of GPUs, the result is nothing short of record-breaking. The recommendations, advice, and code samples in this book will help you pre-train and fine-tune your foundation models from scratch on AWS and Amazon SageMaker while applying them to hundreds of use cases across your organization. With advice from seasoned AWS and machine learning expert Emily Webber, this book helps you learn everything you need to go from project ideation to dataset preparation, training, evaluation, and deployment for large language, vision, and multimodal models. With step-by-step explanations of essential concepts and practical examples, you'll go from mastering the concept of pretraining to preparing your dataset and model, configuring your environment, training, fine-tuning, evaluating, deploying, and optimizing your foundation models. You will learn how to apply the scaling laws to distributing your model and dataset over multiple GPUs, remove bias, achieve high throughput, and build deployment pipelines. By the end of this book, you'll be well-equipped to embark on your project to train and fine-tune the foundation models of the future.
What you will learn.Find the right use cases and datasets for pretraining and fine-tuning.
Prepare for large-scale training with custom accelerators and GPUs.
Configure environments on AWS and SageMaker to maximize performance.
Select hyperparameters based on your model and constraints.
Distribute your model and dataset using many types of parallelism.
Avoid pitfalls with job restarts, intermittent health checks, and more.
Evaluate your model with quantitative and qualitative insights.
Deploy your models with runtime improvements and monitoring pipelines.