In mathematical optimization and decision theory, a loss function or cost function is a function that maps an event or values of one or more variables onto a real number intuitively representing some "cost" associated with the event. An optimization problem seeks to minimize a loss function. An objective function is either a loss function or its opposite, in which case it is to be maximized. The loss function could include terms from several levels of the hierarchy.
Gradient descent
Gradient descent is a method for unconstrained mathematical optimization. It is a first-order iterative algorithm for minimizing a differentiable multivariate function.
The Data Minimization Principle in Machine Learning
The principle of data minimization aims to reduce the amount of data collected, processed or retained to minimize the potential for misuse, unauthorized access, or data breaches. Rooted in...
Backpropagation
In machine learning, backpropagation is a gradient computation method commonly used for training a neural network in computing parameter updates.
media minimization
Media minimization refers to the minimization of the kind of media that is usually high-bandwidth, such as images and videos.
Value engineering
Value engineering (VE) is a systematic analysis of the functions of various components and materials to lower the cost of goods, products and services with a tolerable loss of performance or functionality. Value, as defined, is the ratio of function to cost. Value can therefore be manipulated by either improving the function or reducing the cost. It is a primary tenet of value engineering that basic functions be preserved and not be reduced as a consequence of pursuing value improvements. The term "value management" is sometimes used as a synonym of "value engineering", and both promote the planning and delivery of projects with improved performance.
Reward is not the optimization target — LessWrong
TurnTrout discusses a common misconception in reinforcement learning: that reward is the optimization target of trained agents. He argues reward is b…
WRKSHP.tools | Risk Reward Matrix
The Risk Reward Matrix helps you to balance Risk and Reward when choosing among options.
Leshem (Legend) Choshen 🤖🤗 @ACL @ICML on Twitter / X
I agree, what I say is that you talk about how much you lose of the gradient you compute, and I say, but if you just compute less, the result would lose less and still be equivalent or worse probably?More steps=more information that's true, but a bit of a separate battle right?— Leshem (Legend) Choshen 🤖🤗 @ACL @ICML (@LChoshen) August 17, 2026
Curriculum learning
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).
Functional Decision Theory — LessWrong
Functional Decision Theory is a decision theory described by Eliezer Yudkowsky and Nate Soares, an attempt at a logical decision theory, which says that agents should treat one’s decision as the output of a fixed mathematical function that answers the question, “Which output of this very function would yield the best outcome?”. It is a replacement of Timeless Decision Theory, and it outperforms other decision theories such as Causal Decision Theory (CDT) and Evidential Decision Theory (EDT). For example, it ends with better outcomes than CDT on Newcomb's Problem, ends better than EDT on the smoking lesion problem, and ends better than both in Parfit’s hitchhiker problem. In Newcomb's Problem, an FDT agent reasons that Omega must have used some kind of model of her decision procedure in order to make an accurate prediction of her behavior. Omega's model and the agent are therefore both calculating the same function (the agent's decision procedure): they are subjunctively dependent on that function. Given perfect prediction by Omega, there are therefore only two outcomes in Newcomb's Problem: either the agent one-boxes and Omega predicted it (because its model also one-boxed), or the agent two-boxes and Omega predicted that. Because one-boxing then results in a million and two-boxing only in a thousand dollars, the FDT agent one-boxes. External links: * Functional decision theory: A new theory of instrumental rationality * Cheating Death in Damascus * Decisions are for making bad outcomes inconsistent * On Functional Decision Theory by Wolfgang Schwarz See Also: * Timeless Decision Theory * Updateless Decision Theory * Superrationality * Introduction to Logical Decision Theory for Computer Scientists * Introduction to Logical Decision Theory for Economists * Introduction to Logical Decision Theory for Analytic Philosophers * An Introduction to Logical Decision Theory for Everyone Else
Machine Learning Algorithms, Applications: Key Methods, Real-World Uses
Key Takeaways Traditional algorithms such as regression, decision trees, boosting, clustering, and PCA remain highly useful for practical ML problems.Neural net
Theory of Constraints - The Decision Lab
The theory of constraints is a systems-based methodology that identifies and addresses the weakest link in a process or system.
ME 491: Functional Decomposition
Functional decomposition is a technique for dividing a larger task into smaller tasks. During decomposition, the team focuses on the inputs and outputs of each task and uses generic descriptions of the process achieved by the task. The decomposition process is usually applied recursively until the design team can readily envision ways of achieving each of the sub-tasks. Functional decomposition is also called functional analysis, top-down design, or logical decomposition.
What Are Machine Learning Algorithms? | IBM
A machine learning algorithm is the procedure and mathematical logic through which an AI model learns patterns in training data and applies to them to new data.
lifespan maximization
Lifespan maximization is the extension of hardware lifespan by the users. It may be supported by planned longevity from the manufacturer's side, but it rarely is.
Machine Learning Glossary | Google for Developers
This glossary defines a wide range of machine learning terms, including those specific to TensorFlow and large language models.
