A cost function is a mathematical formula used in optimization and machine learning to measure how well a model performs. It quantifies the difference between the predicted values generated by the model and the actual values from the data. The goal is to minimize this difference, which helps improve the model's accuracy. In the context of linear regression or neural networks, the cost function guides the training process by providing feedback on how to adjust the model's parameters. Common types of cost functions include mean squared error and cross-entropy loss, each suited for different types of problems.