Daniele Mortari

Just another CoE WordPress site

Theory of Functional Connections

Summary

The Theory of Functional Connections (TFC) is a mathematical framework for functional interpolation, transforming constrained optimization problems into unconstrained ones. It achieves this by constructing a constrained functional, a function of a function, that inherently satisfies given constraints regardless of a freely chosen function, thereby generating all possible interpolating functions. This simplifies solving various types of equations, particularly differential equations, and significantly improves the efficiency and accuracy of methods like Physics-Informed Neural Networks (PINNs). TFC offers advantages over traditional methods like Lagrange multipliers and spectral methods by directly addressing constraints analytically and avoiding iterative procedures, although it currently lacks the ability to handle inequality constraints.

Books

  1. Leake, C., Johnston, H, and Mortari, D, “The Theory of Functional Connections: A Functional Interpolation Framework with Applications,” Lulu (2022), ISBN: 9781716816642. The book can be purchased here.

M.S. and B.S. Thesis (4)

  1. De Florio, Mario, Accurate Solutions of the Radiative Transfer Problem via Theory of Connections, MS Thesis in Particles and Radiation Transport, Advisor D. Mostacci, Energy and Nuclear Engineering, University of Bologna (Italy), July 2019.
    ABSTRACT: In this thesis, a new approach to solving a class of radiative transfer problems is presented using the Theory of Functional Connections to solve the linear one-point boundary-value problem derived from the Boltzmann integrodifferential equation for the radiative transfer. The proposed algorithm resides in the category of numerical methods for the solution of transport equations and has been shown to be accurate and suitable for applications in atmospheric science and remote sensing.
  2. Yassopoulos, Christopher, A Comparison of the Finite Element Method and the Theory of Functional Connections with Regards to Solid Mechanics Problems in Engineering, MS Thesis, Advisor J.N. Reddy, Mechanical Engineering, Texas A&M University, Summer 2021.
    ABSTRACT: to be completed.
  3. Gregory Arleth, Application of Theory of Functional Connections for Optimal Control of Nonlinear System, M.S. Thesis, Advisor J. Hurtado, Aerospace Engineering, Texas A&M University, May 2021.
    ABSTRACT: The scope of this research is to show that nonlinear, continuous, and computationally expensive equations can be simplified using ideas from the Theory of Functional Connections. Furthermore, we will show that the form these equations are simplified to are compatible with nonlinear optimal control problem algorithms such as Dynamic Programming. Practical examples involving hypersonic vehicles will also be used.
  4. Pozzi, Chiara, Teoria delle Functional Connections Applicata a Manovre Ottime per Assemblaggio Autonomo in Orbita, BS Thesis, Advisor F. Curti, Laurea in Ingegneria Aerospaziale, Universita’ degli studi di Roma “La Sapienza”, 2019. (In Italian).
    ABSTRACT: In questo elaborato vengono trattate manovre di assemblaggio autonomo di due CubeSat indipendenti ad un loro comune satellite madre. Tali manovre sono ottimizzate in termini di energia e tempo di volo tramite un metodo di risoluzione di equazioni differenziali chiamato \textit{Theory of Functional Connections}. I metodi proposti vengono utilizzati per risolvere il sistema di equazioni differenziali con condizioni al contorno derivante dalla formulazione del metodo indiretto del problema di controllo ottimo, basato sulla funzione Hamiltoniana, e dall’applicazione del Principio di Pontryagin. Il problema \`e stato implementato in MATLAB e sono stati ottenuti risultati molto accurati con tempi computazionali molto brevi (dell’ordine dei 100 millisecondi), fattore che rende l’algoritmo idoneo per future applicazioni in tempo reale.

Ph.D. Dissertations (4)

  1. Johnston, Hunter, The Theory of Functional Connections: A Journey from Theory to Application, Ph.D. Dissertation, Advisor D. Mortari, Aerospace Engineering, Texas A&M University, Summer 2021.
    ABSTRACT: TFC is a general methodology for functional interpolation that can embed a set of user-specified linear constraints. The functionals derived from this method, called \emph{constrained expressions}, analytically satisfy the imposed constraints and can be leveraged to transform constrained optimization problems to unconstrained ones. By simplifying the optimization problem, this technique has been shown to produce a numerical scheme that is faster, more accurate, and robust to poor initialization. The content of this dissertation details the complete development of the Theory of Functional Connections. First, the seminal paper on the Theory of Functional Connections is discussed and motivates the discovery of a more general formulation of the constrained expressions. Leveraging this formulation, a rigorous structure of the constrained expression is produced with associated mathematical definitions, theorems, and proofs. Furthermore, the second part of this dissertation explains how this technique can be used in the solution of ordinary differential equations providing a wide variety of examples with comparison to the state-of-the-art. The final part of this work focuses on unitizing the techniques and algorithms produced in the prior sections to explore the feasibility of using the Theory of Functional Connections to solve problems in real-time optimal control, namely optimal landing problems.
  2. Leake, Carl, The Multivariate Theory of Functional Connections: An n-Dimensional Constraint Embedding Technique Applied to Partial Differential Equations, Ph.D. Dissertation, Advisor D. Mortari, Aerospace Engineering, Texas A&M University, Summer 2021.
    ABSTRACT: This dissertation contains a comprehensive, self-contained presentation of the TFC theory beginning with simple univariate point constraints and ending with general linear constraints in $n$-dimensions; relevant mathematical theorems and clarifying examples are included throughout the presentation to expand and solidify the reader’s understanding. Furthermore, this dissertation describes how TFC can be applied to estimate the solutions of differential equations, its primary application to date. In addition, comparisons with other state-of-the-art algorithms that estimate the solutions of differential equations are included to showcase the advantages and disadvantages of the TFC approach. Lastly, the aforementioned concepts are leveraged to estimate solutions of differential equations from the field of flexible body dynamics.
  3. Schiassi, Enrico, Extreme Theory of Functional Connections fir Problems involving Differential Equations with Applications to Optimal Control Problems, Ph.D. Dissertation, Advisor R. Furfaro, Aerospace and Mechanical Engineering, University of Arizona, November 2021.
    ABSTRACT: In recent years, Neural Networks (NNs) have become widespread across many scientific fields. This requires designing them to satisfy application-specific constraints, such as conservation laws, symmetries, or other domain-specific knowledge. These constraints are usually imposed as soft penalties during network training and act as regularizers within the loss function. Physics-Informed Neural Networks (PINNs) are examples of this philosophy, where the outputs of the network are constrained to approximately satisfy specific physics laws, modeled as a set of Differential Equations (DEs). This dissertation presents a novel, accurate, fast, flexible, reliable, and robust PINN framework for forward and inverse problems governed by DEs. This framework is called Extreme Theory of Functional Connections (X-TFC). The proposed method, for the first time, merges TFC and the Extreme Learning Machines (ELM). TFC enables functional interpolation for a large class of mathematical objects. According to TFC, any mathematical problem solution can be represented via the Constrained Expressions (CEs). The CEs are functional. These functionals are the sum of a free function and a functional that analytically satisfies the problem constraints.
    When applied to problems involving DEs, the mathematical problem is represented by the DE itself, where the constraints are the DE initial and/or boundary conditions. The DE solution is approximated with the CE, where the free function, according to X-TFC, is chosen to be a shallow NN trained via the Extreme Learning Machine (ELM) algorithm. ELM algorithm randomly selects input weights and biases, which are not tuned during the training, leaving the output weights as the only trainable parameters. Thus, the training is performed with a fast and robust least-squares. This work shows the X-TFC advantage over the classic PINN framework and some of its variants. In particular, it will show many X-TFC applications in solving forward and inverse problems involving DEs. X-TFC has been applied to physics-driven solutions of nuclear reactor dynamics (e.g., point kinetic equations with temperature feedback). Another application is for physics-driven solutions of Optimal Control Problems (OCPs), both via the application of the indirect method (e.g., Pontryagin Minimum Principle) and the Bellman Principle of Optimality (BPO), both for general and aerospace OCPs. X-TFC has also been applied for data-physics-driven parameter discovery of epidemiological models that regulate virus spread.
  4. De Florio, Mario, Physics-Informed Neural Networks and Functional Interpolation for Initial Values Problems with Applications to Integro-differential and Stiff Differential Equations, Ph.D. Dissertation, Advisor R. Furfaro, Aerospace and Mechanical Engineering, University of Arizona, November 2022.
    ABSTRACT: This dissertation aims to show the development and adaptation of the Extreme Theory of Functional Connections (X-TFC) framework, a Physics-Informed Neural Network (PINN) combined with Theory of Functional Connections (TFC), to solve Initial Value Problems (IVPs) modeled by differential equations (DEs). In particular, this research focuses on two branches of IVPs: Linear Integro-Differential Equations and Stiff Systems of Non-Linear Differential Equations. The first types of IVPs are DEs where an integral of the unknown function is present, i.e., Integro-Differential Equations. To compute the integral of the unknown function two approaches are used. One is the analytical integration of the unknown function’s approximation, that is represented by the TFC’s constrained expression (CE). The second approximates the definite integral of the unknown function via Gauss-Legendre quadrature. This approach is used to solve problems arising from the Boltzmann Partial Integro-Differential Equation for Transport, such as Radiative Transfer and Rarefied-Gas Dynamics problems. The second types of IVPs are large-scale Stiff Systems of non-linear ODEs, that are mathematical models governing a wide range of real-world problems such as chemistry, biology, epidemiology, engineering, neuroscience, financial systems, and so on. For this reason, recently, there has been a resurgence in the interest in developing new numerical methods capable of solving large time-scale stiff problems. Regular PINNs frameworks are proven to be not accurate (or even not capable) to solve problems when the dynamics are particularly complex (e.g., stiff). This is due to unbalanced back-propagated gradients during the model training. This dissertation shows how X-TFC with a domain decomposition technique overcomes the difficulties of regular PINNs in solving systems of stiff IVPs, and it outperforms the state-of-the-art numerical methods in terms of accuracy and computational times.

Journals (28)

  1. Mortari, D., Garrappa, R., and Nicolo’, L. “Theory of Functional Connections Extended to Fractional Operators,” Functional Interpolation (Mathematics), 2023, , Vol. 11, No. 7.
  2. Mortari, D. “Theory of Functional Connections Subject to Shear-type and Mixed Derivatives,” Functional Interpolation (Mathematics), 2022, 10(24):4692.
  3. Yassopoulos, C., Reddy, J.N., and Mortari, D. “Analysis of Nonlinear Timoshenko–Ehrenfest Beam Problems with von Kármán Nonlinearity using the Theory of Functional Connections,” Mathematics and Computers in Simulation, 2022, In print.
  4. Drozd, K., Furfaro, R., and Mortari, D. “Rapidly Exploring Random Trees with Physics-Informed Neural Networks for Constrained Energy-Optimal Rendezvous Problems,” The Journal of the Astronautical Sciences, 2022. In print.
  5. Mortari, D. “Using the Theory of Functional Connections to solve Boundary Value Geodesic Problems,” Mathematical and Computational Applications, 2022; 27(4):64. https://doi.org/10.3390/mca27040064
  6. D’Ambrosio, A., Schiassi, E., Johnston, H., Curti, F., Mortari, D., and Furfaro, R. “Time-Energy Optimal Landing on Planetary Bodies via Theory of Functional Connections,” Advances in Space Research, 2022, Vol. 69, Issue 12, 15 June 2022, Pages 4198-4220
  7. Mai, T. and Mortari, D. “Theory of Functional Connections Applied to Quadratic and Nonlinear Programming under Equality Constraints,” Journal of Computational and Applied Mathematics, 2022,Vol. 406, 113912.
  8. De Florio, M., Schiassi, E., D’Ambrosio, A., Mortari, D., and Furfaro, R. “Theory of Functional Connections applied to Linear ODEs subject to Integral Constraints and Linear Ordinary Integro-Differential Equations,” Mathematical and Computational Applications, 2021, 26(3), 65. https://doi.org/10.3390/mca26030065
  9. Schiassi, E. De Florio, M., D’Ambrosio, A., Mortari, D., and Furfaro, R. “Physics-Informed Neural Networks and Functional Interpolation for Data-Driven Parameters Discovery of Epidemiological Compartmental Models,” Functional Interpolation (Mathematics), 2021, 9(17), 2069. https://doi.org/10.3390/math9172069.
  10. Yassopoulos, C., Leake, C.,  Reddy, J.N., and Mortari, D. “Analysis of Timoshenko-Ehrenfest Beam Problems using the Theory of Functional Connections,” Special Issue “Computational Approaches to Mechanical Response Analysis of Structures at Diverse Scales,”‘ Journal Engineering Analysis with Boundary Elements, 2021, Vol. 132, pp. 271-280.
  11. Schiassi, E., Furfaro, R., Leake, C., De Florio, M., Johnston, H., and Mortari, D. “Extreme Theory of Functional Connections: A Fast Physics-Informed Neural Network Method for solving Ordinary and Partial Differential Equations,” Neurocomputing, Vol. 457, October 2021, Pages 334-356
  12. Johnston, H., Lo, W.M., and Mortari, D. “A Functional Interpolation Approach to Compute Periodic Orbits in the Circular Restricted Three-Body Problem”, Functional Interpolation (Mathematics), 2021, 9(11):1210. https://doi.org/10.3390/math9111210.
  13. Johnston, H. and Mortari, D. “Least-squares Solutions of Boundary-Value Problems in Hybrid Systems,” Journal of Computational and Applied Mathematics, 393, (2021) 113524.
  14. de Almeida, A., Johnston, H., Leake, C., and Mortari, D. “Fast 2-impulse non-Keplerian orbit-transfer using the Theory of Functional Connections,” The European Physical Journal Plus, 2021, Vol. 136, No. 2, pp. 223.
  15. Drozd, K., Furfaro, R., Schiassi, E., Johnston, H., and Mortari, D. “Energy-Optimal Trajectory Problems in Relative Motion Solved via Theory of Functional Connections,” ACTA Astronautica. Vol. 182, May 2021, pp. 361-382.
  16. Mortari, D. and Furfaro, R. “Univariate Theory of Functional Connections Applied to Component Constraints,” Mathematical and Computational Applications,” 2021,  26(1), 9.
  17. Mortari, D. and Arnas, D. “Bijective Mapping Analysis to Extend the Theory of Functional Connections to Non-rectangular 2-dimensional Domains,” Mathematics 2020, 8(9), 1593; https://doi.org/10.3390/math8091593.
  18. Leake, C.,  Johnston, H., and Mortari, D. “The Multivariate Theory of Functional Connections: Theory, Proofs, and Application in Partial Differential Equations,” Mathematics, 2020, 8(8), 1303.
  19. Furfaro, R. and Mortari, D. “Least-squares Solution of a Class of Optimal Guidance Problems via Theory of Connections,” ACTA Astronautica, March 2020, Vol. 168, pp.92-103.
  20. Leake, C. and Mortari, D. “Deep Theory of Functional Connections: A New Method for Estimating the Solutions of Partial Differential Equations“, Machine Learning and Knowledge Extraction, 2020, Vol. 2, No. 1, pp. 37-55.
  21. Leake, C.,  Johnston, H., and Mortari, D. “The Multivariate Theory of Functional Connections: Theory, Proofs, and Application in Partial Differential Equations,” Mathematics, 2020, 8(8), 1303.
  22. Johnston, H., Leake, C., and Mortari, D. “Least-squares Solutions of Eighth-order Boundary Value Problems using the Theory of Functional Connections“, Mathematics, 2020, Vol. 8(3), No. 397.
  23. Johnston, H., Schiassi, E., Furfaro, R., and Mortari, D. “Fuel-Efficient Powered Descent Guidance on Planetary Bodies via the Theory of Functional Connections,” The Journal of Astronautical Sciences. 2020. DOI: 10.1007/s40295-020-00228-x
  24. Leake, C., Johnston, H., Smith, L., and Mortari, D. “Analytically Embedding Differential Equation Constraints into Least-Squares Support Vector Machines using the Theory of Functional Connections,” Machine Learning and Knowledge Extraction. 2019, 1(4), 1058-1083.
  25. Johnston, H., Leake, C., Efendiev, Y., and Mortari, D. “Selected Applications of the Theory of Connections: A Technique for Analytical Constraint Embedding,” Mathematics, 2019, 7(6), 537.
  26. Mortari, D. and Leake, C. “Multivariate Theory of Connections,” Mathematics, 2019, 7(3), 296.
  27. Mortari, D., Johnston, H., and Smith, L. “High Accurate Least-Squares Solutions of Nonlinear Differential Equations,” Journal of Computational and Applied Mathematics, 2019, Vol. 352, May 15, pp. 293-307.
  28. Mortari, D. “Least-Squares Solutions of Linear Differential Equations,” Mathematics, 2017, 5(4), 48. [2017 Best Paper Award, 2nd place]
  29. Mortari, D. “The Theory of Connections: Connecting Points,” Mathematics, 2017, 5(4), 57.

Conferences (22)

  1. Mortari, D. “The Theory of Connections. Part 1: Connecting Points,” AAS 17-255, 2017 AAS/AIAA Space Flight Mechanics Meeting Conference, San Antonio, TX, February 5-9, 2017.
  2. Mortari, D. “Least-squares Solutions of Linear Differential Equations,” AAS 17-256, 2017 AAS/AIAA Space Flight Mechanics Meeting Conference, San Antonio, TX, February 5-9, 2017.
  3. Mortari, D. “The Theory of Connections with Application,” XVI Jornadas de Trabajo en Mecánica Celeste, Soria (Spain), June 19-21, 2017.
  4. Mortari, D., Johnston, H., and Smith, L. “Least-squares Solutions of Nonlinear Differential Equations,” Paper of the 2018 AAS/AIAA Space Flight Mechanics Meeting Conference, Kissimmee, FL, January 8-12, 2018.
  5. Mortari, D. and Furfaro, R. “Theory of Connections Applied to First-Order System of Ordinary Differential Equations Subject to Component Constraints,” AAS 18-230, 2018 AAS/AIAA Astrodynamics Specialist Conference, Snowbird, UT, August 19-23, 2018.
  6. Johnston, H. and Mortari, D. “Linear Differential Equations Subject to Relative, Integral, and Infinite Constraints,” AAS 18-273, 2018 AAS/AIAA Astrodynamics Specialist Conference, Snowbird, UT, August 19-23, 2018.
  7. Johnston, H. and Mortari, D. “Theory of Connections Solution to Perturbed Lambert’s Problem,” AAS 18-282, 2018 AAS/AIAA Astrodynamics Specialist Conference, Snowbird, UT, August 19-23, 2018.
  8. Furfaro, R. and Mortari, D. “Least-squares Solution of a Class of Optimal Guidance Problems,” AAS 18-362, 2018 AAS/AIAA Astrodynamics Specialist Conference, Snowbird, UT, August 19-23, 2018.
  9. Mortari, D. “The Theory of Functional Connections: Connecting Functions,” IAA-AAS-SciTech-072, Forum 2018, Peoples’ Friendship University of Russia, Moscow (Russia), November 13-15, 2018.
  10. Johnston, H. and Mortari, D. “Weighted Least-Squares Solutions of Over-Constrained Differential Equations,” IAA-AAS-SciTech-081, Forum 2018, Peoples’ Friendship University of Russia, Moscow (Russia), November 13-15, 2018.
  11. Mortari, D. “The Theory of Functional Connections: Current Status,” XIX Colóquio Brasileiro de Dinâmica Orbital, CBDO-2018. Instituto Nacional de Pesquisas Espaciais, São José dos Campos (Brasil), Dec. 3-7, 2018
  12. Drozd, K., Furfaro, R., and Mortari, D. “Constrained Energy-Optimal Guidance in Relative Motion via Theory of Functional Connections and Rapidly-Explored Random Trees,” AAS 19-662, 2019 AAS/AIAA Astrodynamics Specialist Conference, Portland, ME, August 11-15, 2019.
  13. Mortari, D., Mai, T., and Efendiev, Y. “Theory of Functional Connections Applied to Nonlinear Programming under Equality Constraints,” AAS 19-675, of the 2019 AAS/AIAA Astrodynamics Specialist Conference, Portland, ME, August 11-15, 2019.
  14. Schiassi, E., Furfaro, R., Johnston, H., and Mortari, D. “Fuel-efficient Powered Descent Guidance on Planetary Bodies via Theory of Functional Connection,” AAS 19-718, of the 2019 AAS/AIAA Astrodynamics Specialist Conference, Portland, ME, August 11-15, 2019.
  15. Johnston, H., Leake, C., and Mortari, D. “An Analysis of the Theory of Functional Connections Subject to Inequality Constraints,” AAS 19-732, of the 2019 AAS/AIAA Astrodynamics Specialist Conference, Portland, ME, August 11-15, 2019.
  16. Mortari, D. and Leake, C. “An Explanation and Implementation of the Multivariate Theory of Functional Connections via Examples,” AAS 19-734, 2019 AAS/AIAA Astrodynamics Specialist Conference, Portland, ME, August 11-15, 2019.
  17. Johnston, H. and Mortari, D. “Orbit Propagation via the Theory of Functional Connections,” AAS 19-736, 2019 AAS/AIAA Astrodynamics Specialist Conference, Portland, ME, August 11-15, 2019.
  18. Mortari, D. “The Theory of Functional Connections: Current Status,” XXV International Congress of the Italian Association of Aeronautics and Astronautics, Rome (Italy), September 9-12, 2019.
  19. Furfaro, R., Drozd, K., and Mortari, D. “Energy-Optimal Rendezvous Spacecraft Guidance via Theory of Functional Connections,” 70th International Astronautical Congress 2019, IAF Astrodynamics Symposium, Washington, D.C., October 21-25, 2019.
  20. Schiassi, E., D’Ambrosio, A., Johnston, H., De Florio, M., Furfaro, R., Curti, F., and Mortari, D. “Physics-Informed Solution of Optimal Control Problems via Extreme Theory of Functional Connections”, AAS 20-524, Astrodynamics Specialist Conference, August 9-12, Lake Tahoe, CA.
  21. Schiassi, E., D’Ambrosio, A., Johnston, H., Furfaro, R., Curti, F., and Mortari, D. “Complete Energy Optimal Landing on Small and Large Planetary Bodies via Theory of Functional Connections”, AAS 20-557, Astrodynamics Specialist Conference, August 9-12, Lake Tahoe, CA.
  22. Furfaro, R., Schiassi, E., Drozd, K., and Mortari, D. “Physics-Informed Neural Networks and Theory of Functional Connections for Optimal Space Guidance Applications,” IAC 2020, 71-st International Astronautical Congress, 12-16 October 2020, Dubai, United Arab Emirates.