Data-Driven Optimization for Machine Learning Applications - Interactive curriculae of TU Ilmenau

Das Modul Data-Driven Optimization for Machine Learning Applications mit der Prüfungsnummer 220491 schließt mit folgenden Leistungen ab:

• alternative semesterbegleitende Prüfungsleistung mit einer Wichtung von 30% (Prüfungsnummer: 2200829)
• schriftliche Prüfungsleistung über 90 Minuten mit einer Wichtung von 70% (Prüfungsnummer: 2200830)

Details zum Abschluss Teilleistung 1:

Programmieraufgaben als Hausbeleg

Dieses Modul enthält mindestens eine alternative semesterbegleitende Abschlussleistung. Bitte beachten Sie, dass diese in der Regel schon zu Beginn des Semesters, in dem diese angeboten wird, angemeldet werden muss.
Über die Details und Zeiträume dazu werden Sie vom Lehrenden und/oder dem Prüfungsamt informiert. Fragen Sie gegebenenfalls unbedingt beim Lehrenden nach.

This module contains at least one alternative exam part. Please note that this must usually be registered at the beginning of the semester in which it is offered.
The lecturer and/or the examination office will inform you about the details and time periods. If necessary, be sure to ask the lecturer.

 BSc level. Basic linear algebra and computer programming skills are advantageous.

The students know and can explain

• basic model-driven, model-driven data-augumented, and data-driven optimization
• numerical linear algebra methods for machine learning
• convexity and regularization of functions
• non-negative matrix factorization and application
• modern mathematical optimization algorithms for pattern recognition and classification
• modern mathematical optimization algorithms for neural-network-based modeling.

They can implement

• optimization algorithms for linear and nonlinear regressions
• quadratic programming methods for support vector machines
• optimization algorithms for non-negative matrix factorization, pattern recognition, and applications
• and evaluate various optimization algorithms for neural network-based modeling and applications

The students learn the theory, models, methods, and algorithms of the corresponding subjects in the lectures. In the exercises, they are activated to solve example tasks. In project tasks, they analyze, solve, and evaluate programming problems.

1. Introduction - Motivation, Data-driven versus Model-driven appraoch, importance of data-driven optimization; overview of optimization problems arising in machine learning applications;

2. Preiminaries - linear algebra; convex sets convex functions; gradient, sub-gradient, hessian matrix;

3. Programming basics (Python, R, Matlab); data loading and preprocessing; 

4. Unconstrained optimization for machine learning: regularization-meaning and relevance; regression problems; neural networks and back-propagation of errors; optimization methods for deep learning ;

5. Uncostrained Optimiztion Algorithms; 5A: First-order algorithms - gradient descent, accelerated gradient descent, stochastic gradient descent, conjugate gradient methods, coordinate descent; R and Python implementations; sub-gradient methods (optional); 5B. Second-order algorithms: The Newton Method; quasi-Newton methods; LBFGS; R and Python implementations;

6. Constrained Optimization Methods for Machine Learning - the interior point method; face-recongintion with supprot vector machine using Python, Scikit-Learn and OpenCV ;Matrix factorization methods for pattern recognition- SVD, PCA, non-negative matrix factorization (NMF); Matlab and Python Scikit-Learn implementations; Proximal-Point Algorithms: proximal gradient methods; alternating direction of multupliers (ADMM);

7. Bayesian Optimization methods for Machine Learning;

8. Optimization algorithms in Deep Learning Tools TensorFlow, Kerays, pyTorch

Lecture Slides, PC Pools, Machine Learning Tools and Libraries

Bottou, Léon; Curtis Frank E., Nocedal, Jorge: Optimization Methods for Large-Scale Machine Learning. SIAM Review, 60(2), 223-311.                                                                                                                                                                                                                                                                                                                                                                           Emrouznejad, Ali (ed.): Big Data Optimization: Recent developments and challenges. Volume 18, Studies in Big Data Series, Springer, 2016.                                                                                                                                                                          Geron, Aurelien: Hands-on machine learning with scikit-learn, Keras & TensorFlow, 2nd Ed. O'Reilly, 2019.   Goodfellow, Ian; Bengio, Yoshua; Courville, Aaron: Deep Learning. The MIT Press, 2017.