Mini-Batch Gradient Descent
Mini-Batch Gradient Descent is an optimization algorithm used in machine learning to minimize the loss function. It combines the advantages of both Stochastic Gradient Descent and Batch Gradient Descent. Instead of using the entire dataset or just one sample, it processes small random subsets, or "mini-batches," of data. This approach helps to balance the speed of convergence and the stability of the updates.
By using mini-batches, the algorithm can take advantage of vectorized operations, making it more efficient on large datasets. It also introduces some noise into the gradient estimates, which can help escape local minima and improve generalization in models like neural networks.