Key takeaways
Optimization is the process of finding the best solution from all possible choices. Machine learning optimization is the mathematical process of adjusting the internal parameters “θ” of a model, such as the weights and biases of a neural network. The adjustment minimizes the loss function L(θ), which measures prediction error on the training data. Gradient-based algorithms, such as stochastic gradient descent (SGD) and Adam, are used to train neural networks. Derivative-free methods, such as Bayesian optimization and genetic algorithms, apply to expensive or non-differentiable objectives.
High computational costs limit the number of iterations in simulation-driven design. Each high-fidelity computer-aided engineering (CAE) simulation, such as a computational fluid dynamics (CFD) or finite element analysis (FEA) run, takes hours or days (Microsoft; reference 1). In engineering, machine learning optimization also encompasses optimizing products using a trained model as part of broader simulation-driven design methodologies. The trained model replaces the solver for evaluating candidate designs. This replacement lowers evaluation costs and shortens design cycles. Engineers can therefore evaluate a wider range of options before final selection.
This article is written for practicing engineers, data scientists, and engineering managers. For CAE, CFD, and FEA engineers integrating machine learning into simulation workflows, it provides the mathematical details needed for verification. For data scientists working with engineering teams, it connects model optimization to design optimization. For engineering managers evaluating the return on optimization efforts, it provides a case study with measured results.
The article clarifies two related but distinct aspects of optimization: machine learning optimization and engineering optimization via machine learning. Model optimization improves machine learning performance through hyperparameter tuning, feature selection, architecture design, and training refinement. Engineering optimization couples reliable predictive models with consolidated optimization techniques for innovative product design. The sections cover loss functions, gradient-based and adaptive optimizers, Bayesian optimization, metaheuristics, 3D shape optimization, and an industrial case study. Reinforcement learning and model compression appear only in the FAQ. Implementation code and solver benchmarks are outside its scope.
Table 1. Mathematical notation and meaning for the Article. Readers may refer back to this table if any mathematical notation is unfamiliar.
| Math | Meaning in plain English |
|---|---|
| ŷᵢ = f(xᵢ; θ) | The model “f” maps the input xᵢ of sample “i” to the prediction ŷᵢ. The parameters θ, the network’s weights and biases, determine this mapping. |
| L(θ) = (1/N) Σᵢ₌₁ᴺ ℓ(f(xᵢ; θ), yᵢ) | The cost function averages the loss ℓ over the “N” training samples. Each loss term measures the deviation of one prediction from its known target yᵢ. |
| θ* = arg min L(θ) | Training returns the parameter values θ* that minimize the cost function. The operator arg min gives the location of the minimum; min L(θ) would give its value. |
| MSE = (1/N) Σᵢ (ŷᵢ − yᵢ)² | Mean squared error: the average of the squared prediction errors. Squaring gives large deviations more weight than small ones. |
| MAE = (1/N) Σᵢ |ŷᵢ − yᵢ| | Mean absolute error: the mean of the absolute differences between the predicted and actual values. |
| RMSE = √MSE | Root mean squared error: the square root of the MSE. The result has the unit of the target quantity, such as pascals for a pressure field. |
| Cross-entropy = −(1/N) Σᵢ Σₖ yᵢₖ log ŷᵢₖ | Loss for classification into “K” classes. The target yᵢₖ equals 1 for the true class of sample “i” and 0 for the other classes; ŷᵢₖ is the predicted probability of class “k.” The loss increases as the predicted probability of the true class decreases. |
| R² = 1 − Σᵢ (yᵢ − ŷᵢ)² / Σᵢ (yᵢ − ȳ)² | Fraction of the variance of the targets explained by the model. R² = 1 corresponds to a perfect fit; R² = 0 corresponds to a model that always predicts the mean ȳ. |
| L(θ) + λ‖θ‖² | Regularized cost: the cost function plus a penalty on the squared magnitude of the parameters. The weight λ ≥ 0 balances the fit to the data and the magnitude of the parameters. |
| θₜ₊₁ = θₜ − η ∇L(θₜ) | One gradient descent step. The gradient ∇L(θₜ) points in the direction of steepest increase of the cost. The update subtracts this gradient, scaled by the learning rate η, from the current parameters θₜ. |
| minimize J(p) subject to gⱼ(p) ≤ 0 | Design optimization problem. The solution is the set of design variables “p” with the lowest objective J(p), such as drag or mass, among the designs that satisfy every constraint gⱼ(p) ≤ 0, such as a stress limit. |
Optimization theory examines algorithms and strategies for identifying the optimal solution to a problem. The goal of optimization is to minimize or maximize a specific objective, expressed as an objective function: a measure of how good or bad a solution is. In supervised learning, the objective function is the loss function, also called the cost function.
A supervised model “f” maps an input “x” to a prediction “ŷ.” The parameters “θ,” the weights and biases of a neural network, determine this mapping:
ŷᵢ = f(xᵢ; θ)
For each training sample “i,” the loss ℓ(ŷᵢ, yᵢ) quantifies the deviation of the prediction ŷᵢ from the known target yᵢ. The cost function averages the loss over the entire dataset of “N” training samples:
L(θ) = (1/N) Σᵢ₌₁ᴺ ℓ(f(xᵢ; θ), yᵢ)
Training determines the parameter values θ* at the minimum of the cost function, with “n” the number of parameters:
θ* = arg min L(θ), θ ∈ ℝⁿ
Many texts use “loss” for a single data point and “cost” for the average over all data points; others use loss function, cost function, and objective function interchangeably. In supervised learning, the known targets yᵢ make the cost function computable for any parameter vector “θ.” Two models trained on the same data can therefore be compared by their loss on a common validation set.
The training loop repeats prediction, loss evaluation, gradient computation, and parameter update until the loss on a validation set reaches its minimum.
There are several ways to express the loss function mathematically.
With eᵢ = ŷᵢ − yᵢ, the error on sample “i,” the main expressions are:
For fields predicted over a discretized geometry, such as a pressure field on the Mᵢ surface nodes of design “i,” the squared error can also be averaged over the nodes:
L(θ) = (1/N) Σᵢ₌₁ᴺ (1/Mᵢ) Σₘ (ûᵢₘ − uᵢₘ)²
In this expression, uᵢₘ and ûᵢₘ denote the simulated and predicted field values at node “m.”
Machine learning optimization differs from pure mathematical optimization in its goal (Goodfellow, Bengio and Courville, 2016; reference 2). Pure optimization minimizes a known objective as a goal in itself. In machine learning, the relevant measure is the error on data outside the training set. The training loss L(θ) is only an indirect proxy for this generalization error. A model can achieve a low training loss but high error on new data. This condition is called overfitting.
Regularization reduces overfitting through a penalty on the magnitude of the parameters:
L̃(θ) = L(θ) + λ ‖θ‖₂²
The weight λ ≥ 0 controls the trade-off between fitting the training data and the magnitude of the parameters. The L2 penalty λ‖θ‖₂², also called weight decay, reduces the magnitude of all parameters; the L1 penalty λ‖θ‖₁ produces sparse parameter vectors, with many parameters equal to zero (reference 2). Early stopping ends the training when the loss on a validation set reaches its minimum.
Regularization thus expresses a preference for small parameters by adding a soft penalty to the loss. In engineering design problems, requirements are hard constraints instead.
In engineering design optimization, the variables are the design variables “p,” such as shape parameters or wall thicknesses. The design objective J(p) is a performance quantity, such as drag or mass. The requirements take the form of hard constraints:
minimize J(p) subject to gⱼ(p) ≤ 0 and hₖ(p) = 0
The inequality constraints gⱼ(p) ≤ 0 bound quantities, such as stress or displacement. The equality constraints hₖ(p) = 0 fix quantities, such as the volume of the part or the position of a mounting interface. When a trained model evaluates the objective and constraints, its prediction replaces the CFD or FEA simulation at each design point. The quality of the optimized design then depends on the trained model’s accuracy within its training range.
Supervised learning and design optimization are both optimization problems. Supervised learning minimizes a prediction error, whereas design optimization determines the best design given specific performance objectives and constraints. The two uses are compared in Table 2.
Table 2. Model optimization and design optimization compared
| Aspect | Model optimization | Design optimization |
|---|---|---|
| Variables | Parameters θ | Design variables p |
| Objective | Loss L(θ) | Objective J(p) |
| Constraints | Soft, via regularization | Hard, gⱼ(p) ≤ 0 |
| Evaluation | Training data | Trained model |
| Algorithms | SGD, Adam | Gradient-based, Bayesian, genetic |
| Result | Trained model f(x; θ*) | Optimized design p* |
Machine learning-based optimization applies machine learning (ML) techniques to product and process optimization across various engineering domains.
Traditional optimization methods become inefficient in high-dimensional, non-convex, or computationally expensive design spaces.
As the number of dimensions increases, the search space grows exponentially. For instance, a grid with “k” levels for each of “d” variables contains kᵈ points: 10 levels for 6 variables give 10⁶ = 1,000,000 evaluations. Exhaustive exploration is therefore infeasible for finding optimal hyperparameters or design variables.
Many real-world problems are also non-convex. Their objective functions have multiple local minima, at which gradient-based methods can stop before reaching the global optimum. Evaluating complex CAE simulations, such as CFD and FEA, for every iteration is expensive and slow.
ML models trained on simulation data provide physics-aware AI predictions. During optimization, these predictions take the place of most costly finite element analyses or computational fluid dynamics simulations, while the solver verifies the best candidates. Moreover, each run of a traditional method evaluates its designs independently of earlier runs. In contrast, ML models trained on earlier simulations generalize to new designs within their training range. This generalization accelerates future searches.
Aerodynamic shape optimization illustrates the computational cost of traditional algorithms: each simulation-based evaluation of a design iteration takes hours or days (Microsoft; reference 1). Applying machine learning in CFD to accelerate simulations lets an AI model trained on high-fidelity CAE data provide predictions in seconds or milliseconds, at a fraction of the cost of running expensive simulations for every evaluation (reference 1).
ML models, such as Gaussian processes, neural networks, and 3D deep learning models, approximate the objective function J(p). In aerospace, these AI-driven design approaches reduce computational cost and guide the optimization toward high-performance designs.
Neural Concept’s Engineering Intelligence Platform works directly on 3D geometry, which minimizes the need for manual simplifications: its Geometry Intelligence represents complex spatial features of the geometry directly. This direct representation increases the fidelity of the optimization process.
Examples include using machine learning to optimize aerodynamic profiles in aerospace or F1 design, structural topologies in mechanical engineering, thermal management systems in electronics, and manufacturing process parameters in materials science.
These machine learning-driven approaches combine supervised learning techniques with optimization algorithms. The resulting hybrid solutions account for the uncertainty in real-world engineering problems and the constraints typical in engineering design.
ML models can also be embedded directly in user-facing tools. Engineers and end users then optimize designs manually or through automated algorithms.
Machine learning optimization refers to the process of improving the performance of machine learning models. The process addresses the following aspects:
The goal is higher accuracy, faster convergence, and lower error. Optimization also reduces overfitting and shortens training time.
Hyperparameters are settings chosen before training a machine learning model, whereas parameters are fitted to the data. These settings control aspects such as model complexity and learning efficiency, thereby influencing performance. Proper tuning of hyperparameters can reduce prediction errors by ensuring the model generalizes well to unseen data. For example, the learning rate “η” in a neural network (NN) determines how much the model weights change per iteration. A high rate causes instability, while a low rate slows convergence.
Common hyperparameter tuning approaches include grid search and random search. The approaches differ in cost and efficiency:
All these techniques minimize an objective: the loss L(θ) during training and a validation loss during hyperparameter tuning. Each method is suited to different scenarios, depending on the nature of the data, the model’s architecture, and the available computational resources. The choice of optimization method affects the training speed, the quality of the final model, and its ability to generalize.
Before optimization, data preprocessing, including feature scaling, ensures that input parameters contribute proportionally to the optimization process. For instance, standardization computes x̃ⱼ = (xⱼ − x̄ⱼ) / sⱼ, with x̄ⱼ and sⱼ the mean and standard deviation of input “j.” This step improves the numerical stability of models, such as Gaussian processes and deep learning models. This stability increases the accuracy of their design predictions.
The gradient descent method illustrates the basic mathematics of optimization, common to machine learning and engineering problems. Gradient descent is one of the first-order optimization algorithms used in machine learning. It is an iterative method for finding a local minimum of a differentiable multivariate function.
The optimization variables depend on the application:
The loss landscape is a terrain map showing how the loss L(θ) varies with the parameters “θ”: the height of each point represents the loss value. The minimum of L(θ) corresponds to the lowest point of the terrain. The gradient ∇L(θ) is the vector of the partial derivatives ∂L/∂θⱼ. It indicates the direction in which L(θ) increases the fastest at the current point.
This property leads to the gradient descent update rule:
θₜ₊₁ = θₜ − η ∇L(θₜ)
Here, “t” denotes the iteration index; the learning rate “η” is the factor controlling the step size. The negative sign sets the direction of each update opposite to the gradient. The updates are repeated until “θ” converges to a minimum of L(θ). Iterative optimization involves comparing the model’s output with the expected results after each iteration.
In engineering, the same rule updates the design variables, pₜ₊₁ = pₜ − η ∇J(pₜ), when the objective J(p) is differentiable with respect to “p.”
The learning rate “η” is a hyperparameter of gradient descent. It controls the step size for each parameter update and, therefore, the rate at which the loss L(θ) decreases.
Some optimization techniques, such as Adam and RMSprop, dynamically adjust the learning rate during training. These methods scale each parameter’s step inversely to the magnitude of its recent gradients. This scaling balances speed and stability.
For a given training dataset, the accuracy of the trained model depends on the choice of learning rate. A common strategy is to start with a moderate learning rate and to reduce it over the course of training. This reduction, called learning-rate decay, fine-tunes the solution as the model nears convergence. For reference, Keras, an open-source deep learning library, sets 0.01 as the default learning rate of its SGD optimizer (Keras; reference 4).
A loss landscape shows how a model’s loss varies with its parameters. Each set of parameter values has a height equal to its loss; the result is a surface, also called the optimization landscape. The goal of training is the lowest point of this surface.
For instance, a house-price model takes inputs, such as size, number of bedrooms, and location. For any given parameter setting, the loss is the gap between predicted and actual prices across the dataset.
Each gradient descent update adjusts the parameters in the direction of the negative gradient. The updates continue until the decrease in loss falls below a threshold or another stopping criterion is met. Momentum and adaptive learning rates adjust the step size over time. This adjustment accelerates convergence and reduces the risk of stopping in a shallow local minimum.
The shape of the optimization landscape determines the difficulty of the search. In convex optimization problems, each local minimum is also a global minimum (Boyd and Vandenberghe, 2004; reference 5). The loss landscapes of deep neural networks are non-convex, with many local minima and saddle points (reference 2). A gradient-based search can end at any of these points.
Stochastic gradient descent (SGD) is an extension of gradient descent with random sampling of the training data. SGD computes the gradient and updates the model parameters using individual data points or small batches of training examples:
θₜ₊₁ = θₜ − (η/|B|) Σᵢ ∇ℓ(f(xᵢ; θₜ), yᵢ)
Here, the sum runs over the |B| samples of the mini-batch “B.” Each update uses a small part of the data. This sampling reduces memory usage and improves computational efficiency for large datasets. The gradient noise also lets SGD escape local minima more effectively than full-batch gradient descent: SGD effectively operates on a smoothed version of the loss (Kleinberg, Li, and Yuan, 2018; reference 21).
Mixed-precision training further reduces memory use: weights, activations, and gradients are stored as 16-bit floating-point numbers, with a 32-bit master copy of the weights for updates (Micikevicius et al., 2018; reference 22). The authors report that this scheme nearly halves memory requirements.
Adagrad adapts the learning rate for each parameter (Duchi, Hazan and Singer, 2011; reference 6). With gₜ = ∇L(θₜ), the update divides each step by the root of the accumulated squared gradients:
θₜ₊₁ = θₜ − η gₜ / √(Gₜ + ε), with Gₜ = Σₛ₌₁ᵗ gₛ²
All operations are applied element-wise, with a small constant “ε” to prevent division by zero. For parameters associated with infrequent features, the sum Gₜ remains small. Their updates are therefore larger than those of frequent features. This property makes Adagrad effective for sparse data, where each sample has few nonzero features.
RMSprop replaces the sum Gₜ with a moving average of squared gradients (Tieleman and Hinton, 2012; reference 7):
vₜ = β vₜ₋₁ + (1 − β) gₜ²; θₜ₊₁ = θₜ − η gₜ / √(vₜ + ε)
The decay rate “β” limits the influence of old gradients. This limit prevents the effective learning rate from decreasing too rapidly.
Adam (adaptive moment estimation) is an optimizer used to train deep neural networks (Kingma and Ba, 2015; reference 8). The Adam optimizer combines momentum and RMSprop; Kingma and Ba describe it as computationally efficient, with little memory requirement (reference 8). Adam adapts the learning rate of each parameter based on historical gradients: it computes moving averages of the gradient and of its square.
The moving averages are:
mₜ = β₁ mₜ₋₁ + (1 − β₁) gₜ; vₜ = β₂ vₜ₋₁ + (1 − β₂) gₜ²
After the bias corrections m̂ₜ = mₜ / (1 − β₁ᵗ) and v̂ₜ = vₜ / (1 − β₂ᵗ), the parameter update is:
θₜ₊₁ = θₜ − η m̂ₜ / (√v̂ₜ + ε)
Kingma and Ba give η = 0.001, β₁ = 0.9, β₂ = 0.999, and ε = 10⁻⁸ as good default settings for the tested machine learning problems (reference 8). They also describe the method as appropriate for problems with very noisy or sparse gradients. Their tests include a natural language processing problem with sparse features: logistic regression on bag-of-words vectors of IMDB movie reviews (reference 8).
Second-order methods use the curvature of the loss in addition to its gradient. The curvature is described by the Hessian matrix H = ∇²L(θ), the n × n matrix of the second derivatives ∂²L/∂θⱼ∂θₖ. Newton’s method scales the gradient step with the inverse Hessian:
θₜ₊₁ = θₜ − H⁻¹ ∇L(θₜ)
On smooth problems near a minimum, Newton’s method converges in fewer iterations than gradient descent (Nocedal and Wright, 2006; reference 9). For “n” parameters, however, the Hessian has n² entries. This size makes the exact Hessian impractical for models with millions of parameters.
Quasi-Newton methods, such as BFGS (named after Broyden, Fletcher, Goldfarb, and Shanno), approximate the inverse Hessian from successive gradients. L-BFGS, the limited-memory variant, stores a few recent vector pairs instead of the full matrix. The method was designed for large-scale optimization (Liu and Nocedal, 1989; reference 10). First-order methods, such as SGD and Adam, remain the default for deep networks, where the number of parameters and gradient noise make curvature estimates costly and unreliable (reference 2).
Bayesian optimization provides a rigorous framework for optimizing expensive-to-evaluate objective functions J(p) by maintaining and updating probability distributions over candidate solutions. In this section, “p” denotes the search variables: the hyperparameters of a model or the design variables of a product.
The method constructs a probabilistic model of the objective, typically a Gaussian process. The model provides a predicted value μ(p) and an uncertainty σ(p) at any point “p” of the search space. At each iteration, an acquisition function selects the next point to evaluate from μ(p) and σ(p). One acquisition function is the expected improvement over the best value Jₘᵢₙ observed so far (Jones, Schonlau and Welch, 1998; reference 11):
EI(p) = E[max(0, Jₘᵢₙ − J(p))]
The expectation applies to the predictive distribution of J(p). The expected improvement is large at points with a low predicted value or a high uncertainty. The result of each new evaluation updates the probabilistic model; the cycle repeats until the evaluation budget is exhausted.
Metaheuristic optimization algorithms guide lower-level heuristic techniques in exploring complex search spaces. Many are modeled on natural processes, such as evolution for genetic algorithms or bird flocking for particle swarm optimization. These algorithms search for optimal solutions to problems in engineering and logistics. When evolutionary algorithms tune hyperparameters, each candidate set of hyperparameter values is evaluated. The highest-scoring sets are then combined to form new candidates.
Genetic algorithms are modeled on natural evolution (Holland, 1975; reference 12). Each generation is a population of candidate solutions, encoded as chromosomes. An objective function evaluates each candidate; selection retains the best chromosomes. Combination (crossover) and mutation then generate new solutions from the selected chromosomes until a satisfactory solution is found.
Each generation requires one objective evaluation per candidate. When objectives are simulation-based, this requirement makes genetic algorithms computationally demanding.
Swarm intelligence algorithms reproduce the collective behavior of organisms to solve optimization problems. Each agent follows simple rules. The interaction of many agents produces a collective search of the design space.
Optimization algorithms are classified into families based on the information used at each step. Table 3 summarizes the families described in this section.
Table 3. Families of optimization algorithms
| Family | Information used | Examples | Cost per step | Typical use |
|---|---|---|---|---|
| First-order | Gradient | SGD, Adam, RMSprop | One gradient per mini-batch | Deep networks |
| Second-order | Gradient and curvature | Newton, BFGS, L-BFGS | Gradient plus curvature update | Smooth, deterministic problems |
| Derivative-free | Objective values | Bayesian, genetic, swarm | One evaluation per candidate | Expensive or non-differentiable objectives |
The choice of family depends on the cost of an evaluation. When gradients are available at low cost, first-order methods scale to millions of parameters. When each evaluation requires a simulation, Bayesian optimization reduces the number of evaluations. Genetic algorithms and swarm methods apply to non-differentiable objectives, at the price of many evaluations.
In the design of industrial objects, engineers optimize shapes in CAD for performance, materials, and manufacturability. For instance, optimizing a car’s rear window for aerodynamics or a lightweight bracket for fatigue resistance requires simulations. CAE simulations assess performance in fluid dynamics or structural analysis. A simulation campaign can take weeks and demand high computing power. As a documented example, pedestrian-safety evaluations at General Motors required weeks of simulations before the introduction of an AI model (Neural Concept, 2025; reference 16).
Companies accelerate CAE through investments in High-Performance Computing (HPC), GPU-based parallel processing, or AI models for real-time prediction of simulation results, often coupled with high-quality meshing of 3D geometries to ensure accurate CFD and FEA analyses.
The computational cost of CAE limits the number of analyses per day. This limit creates a need for real-time alternatives, such as physics-aware AI models. Deep learning provides these alternatives. In the Neural Concept platform, neural networks (NNs) extract geometric features directly from 3D shapes.
In AI and data science, predictive models are trained on consistent datasets, such as temperature and pressure data from past simulations. Training minimizes the NN’s prediction error. The trained network then links design variables with performance goals.
Deep learning (DL) depends on data: training data influence a model’s ability to recognize patterns and make predictions. Larger datasets typically improve performance. Data quality also affects performance: poor or biased data reduce the reliability of the results.
Traditional AI systems apply rule-based decisions, whereas DL models provide predictions. Final decisions remain with the users, even as optimization capabilities evolve.
Training data affect model performance, generalization, and accuracy. High-quality, diverse data improve pattern recognition, reduce bias, and support reliable predictions. In contrast, poor data can result in overfitting, underfitting, or biased outputs. These defects hinder real-world use.
For instance, convolutional neural networks (CNNs) trained on a limited number of medical images may generalize poorly to new scans. This limitation reduces diagnostic reliability. Similarly, self-driving car models trained only on images of sunny conditions may produce errors in rain or snow. When data are scarce, engineers use strategies, such as synthetic data generation or transfer learning.
CNNs are particularly relevant in engineering applications because they can process and analyze complex image-based data. Their convolutional layers extract spatial features from 2D images, 3D models, and CAD designs. This capability makes CNNs effective for engineering tasks.
Industrial applications, such as defect detection, have succeeded with small datasets. For instance, Tabernik and colleagues trained a segmentation-based network for surface-defect detection with about 25 to 30 defective training samples (Tabernik, Šela, Skvarč and Skočaj, 2020; reference 17). In CAD engineering simulations, initial data from past designs and carefully planned AI model data collection and generation can reduce future data requirements.
Testing new designs through simulations is time-consuming and costly. Design spaces have many dimensions, such as geometric parameters and material properties. This dimensionality makes manual optimization impractical. For example, optimizing a car’s aerodynamics involves tuning interrelated shape parameters.
Such problems require a constrained optimization algorithm. The algorithm balances the objectives of structural strength, weight reduction, and energy efficiency. It also satisfies the equality and inequality constraints of the design problem, including limits on material and manufacturing costs. In non-convex optimization problems, saddle points can stop gradient-based searches. The algorithm also handles convex constraints in such problems.
Designers optimize 3D shapes from a digital product shape in CAD. Functional requirements, such as weight and mechanical robustness, define the objectives. Standard methods connecting CAD to optimization algorithms include:
In machine learning-based shape optimization, the artificial NN in the AI software is trained to predict, for example, the pressure field over the surface of a wing. The aerodynamic coefficients of the wing follow from the integration of this field over the surface. The three-dimensional optimization based on machine learning then modifies the mesh at each iteration. The shape evolves until it reaches an optimal form under the given constraints.
For instance, Baqué and colleagues trained geodesic convolutional neural networks to emulate a fluid dynamics simulator (Baqué, Remelli, Fleuret and Fua, 2018; reference 18). The trained network provided an objective function differentiable with respect to the shape parameters. A gradient-based technique then optimized the shape parameters.
Periodic high-fidelity CFD simulations verify the candidate designs. The verified cases can then be used to extend the model’s training data.
The machine learning-based optimization of aerodynamic profiles in aerospace design is an example of AI’s use in mechanical engineering. Neural Concept, the drone manufacturer senseFly, and the simulation provider AirShaper applied it to a fixed-wing drone.
The case describes the collaboration between data scientists and engineers to develop a higher-performing industrial design. A deep learning model trained on the team’s simulations evaluated each candidate design.
The objective was to maximize the drone’s lift-to-drag (L/D) ratio, with a minimum lift of 8.0 N (Neural Concept; reference 19). The L/D ratio affects the autonomy of an unmanned aerial vehicle (UAV). The optimization process used simulation software (CAE) to evaluate UAV performance across configurations.
Neural Concept combined CFD simulations with AI predictions to explore and optimize drone aerodynamics. The workflow used the following levels of CFD fidelity:
The framework began with low-accuracy simulations for pre-training and was later refined using high-accuracy simulations. Low-fidelity CFD and the pre-trained NN supported innovative design exploration. Pre-training on the low-fidelity results clearly improved what the network could learn from a budget of only 50 high-fidelity samples (reference 19).
The final optimization used 40 detailed simulations. The approach increased the L/D ratio by 4.25% and reduced drag by 6.25% at maximum L/D (reference 19).
