• Table of contents
• Wiki of the Machine Learning / Deep Learning Pole
  • Seminars
  • Reference publications
  • Books & Code Examples
  • Frameworks to start / to document in ML/DL
  • Online courses & training
  • Available GPUs computing resources for LAM staff
  • Workshops / Conferences
  • Initial meetings (in French)

Wiki of the Machine Learning / Deep Learning Pole¶

INFORMATION PAGE for the ML/DL Pole at CeSAM.
This is a selection of some references that may be useful to start or consolidate your knowledge. This information is of course not exhaustive and any suggestions for additions are welcome. If you have any questions, please send an email to morgan.gray@lam.fr

Seminars¶

For the presentations, the slides can be retrieved with the link; the videos can be viewed on https://seminars.lam.fr/#MLDL

• 04/05/2021 : Raoul Canameras (MPIA-Garching) "Finding and modeling strong gravitational lenses with deep neural networks"
• 30/03/2021 : Alexandre Boucaud & Hubert Bretonnière (LAC) "FlowVAE: taking control of galaxy image simulations with deep generative networks"
• 09/03/2021 : Laurent Risser (IMT) "Explainability techniques for black-box decision rules in Machine Learning" (210309 Risser)
• 11/02/2021 : Sidonie Lefebvre (ONERA/DOTA) "Generative Adversarial Networks (GANs) : concept and application to cloudy sky images synthesis" (210211 Lefevbre)
• 01/02/2021 : Nicolas Audebert (CNAM) "Hyperspectral remote sensing data analysis using Deep Learning" (210201 Audebert)
• 18/01/2021 : François-Xavier Dupé (LIS/QARMA) "How Machine Learning can help to automate processing tasks ? An example with image denoising" (210118 Dupe)
• 11/01/2021 : Julien Wojak (Institut Fresnel) "Deep Learning : focus on auto-encoder as a pre-processing step for classification" (210111 Wojak)

Reference publications¶

• General comments on :
  • Machine Learning : Baron(19) MLinAstronomyOverview & Mehta MLforPhysicists
  • Deep Learning : Hadji(18) CNN-Overview
• Some classic Neural Network architectures
  • VGGNet : Simonyan VGGNet ; ResNet : He(15) ResNet & He(16) ResNet
  • Inception : Szegedy(14) Inception, Szegedy(15) RethinkingInception & Szegedy(16) InceptionV4 InceptionResNet
  • Xception : Chollet Xception
  • Generative Adversarial Networks : Goodfellow GenerativeAdversarialNets
  • Adversarial Auto Encoders : Makhzani AdversarialAutoencoders
• Practical advice (methodology, specific aspects)
  • Hyperparameters utilization : Bengio(12) PracticalRecommendations & Smith(18) Hyperparameters
  • Optimizers : OptimizerAlgorithms ; Dropout : Dropout ; Batchnormalisation : BatchNormalization
• Publications (research in ML/DL)
  • Papers with Code: https://paperswithcode.com/

Books & Code Examples¶

• Hands-On Machine Learning with Scikit-Learn, Keras & Tensorflow by Aurélien Géron (O'Reilley editor, second edition)
  • Jupyter notebooks for the code samples : https://github.com/ageron/handson-ml2
• Deep Learning with Python by François Chollet (Manning editor) : a good presentation of Keras to make Deep Learning
  • Jupyter notebooks for the code samples : https://github.com/fchollet/deep-learning-with-python-notebooks
  • Examples of codes with different problems & neural networks : https://keras.io/examples/ ; https://keras.io/api/applications/
• Deep Learning by Goodfellow, Bengio, Courville (MIT press book editor) : a more theoretical book, free access per chapter https://www.deeplearningbook.org/
• Pattern Recognition & Machine Learning by Christopher M. Bishop (Springer Editor)
• Statistics, Data Mining and Machine Learning in Astronomy by Ivezic, Connolly, Vanderplas & Gray (Princeton Press)
  • HTML documentation : https://www.astroml.org/ ; Source-code repository: https://github.com/astroML/astroML
• The Elements of Statistical Learning by Hastie, Tibshirani & Friedmman (Springer Editor) : free to download https://web.stanford.edu/~hastie/ElemStatLearn/

Frameworks to start / to document in ML/DL¶

• ScikitLearn : https://scikit-learn.org/stable/user_guide.html
• Keras : https://keras.io/api/ ou https://keras.io/getting_started/intro_to_keras_for_engineers/
• Tensorflow : https://www.tensorflow.org/api_docs/python/tf
• Pytorch : https://pytorch.org/get-started/locally/ ; https://pytorch.org/tutorials/
• MLflow, an Open Machine Learning Platform : https://mlflow.org/

Online courses & training¶

• FIDLE 2021 (in French) : Introduction au Deep Learning (accès 100% libre, aucune inscription, tout est libre, se connecter 5 min avant !)
  • En bref : http://bit.ly/fidle-a-distance ; Programme : http://bit.ly/fidle-a-distance-programme
  • supports : https://gricad-gitlab.univ-grenoble-alpes.fr/talks/fidle
  • videos : https://www.youtube.com/channel/UC4Sukzudhbwr6fs10cXrJsQ
• JDEV 2020 (in French) :
  • Python, préparation des données avec Panda & Apprentissage Automatique avec Scikitlearn : http://devlog.cnrs.fr/jdev2020/t8.a05 ; supports https://gitlab.in2p3.fr/aboucaud/atelier-jdev-2020
  • Fondamentaux Apprentissage Automatique avec Python & ScikitLearn : http://devlog.cnrs.fr/jdev2017/t8.a04 ; supports http://laurent.risser.free.fr/TMP_SHARE/JDEV2020/
  • Introduction Keras & Tensorflow : http://devlog.cnrs.fr/jdev2020/t8.ap04 ; supports https://gricad-gitlab.univ-grenoble-alpes.fr/talks/fidle/
  • Introduction Deep Learning & PyTorch : http://devlog.cnrs.fr/jdev2020/t8.ap01 ; supports http://laurent.risser.free.fr/TMP_SHARE/JDEV2020_T8_AP01/
• CNAM (in French) :
  • RCP208 : Reconnaissance des formes et méthodes neuronales (http://cedric.cnam.fr/vertigo/Cours/ml/)
  • RCP209 : Apprentissage, réseaux de neurones et modèles graphiques (http://cedric.cnam.fr/vertigo/Cours/ml2/)
  • Cours de Nicolas Thome : slides Formation CNAM ; videos : send an email request
• ML/DL Mastery (in English) : https://machinelearningmastery.com/start-here/
• KAGGLE (in English) : https://www.kaggle.com/learn/overview ; https://www.kaggle.com/learn/intro-to-machine-learning ;
  https://www.kaggle.com/learn/intro-to-deep-learning ; https://www.kaggle.com/learn/computer-vision
• STANFORD University (in English) :
  • CNN for Visual Recognition : https://cs231n.github.io/ ou http://cs231n.stanford.edu/syllabus.html
  • COURSERA : slides Formation COURSERA ; videos https://www.coursera.org/learn/machine-learning#syllabus

Available GPUs computing resources for LAM staff¶

• LAM cluster (responsible JC Lambert) : 7 GPUs nvidia RTX2080Ti (single precision, 11 GB) + 1GPU TitanXP
  https://projets.lam.fr/projects/cluster-de-calcul-du-lam/wiki#GPU-partition
• Aix-Marseille University Mesocentre cluster : different partitions (for the volta partition: 20 GPUs nvidia V100, double precision, 32 GB)
  https://mesocentre.univ-amu.fr/appel-a-projets/
• How to submit a job on the Mesocentre cluster
  • in /home/myaccount/ :
    copy the shell script source setup Meso and put it in this directory
  • in /home/myaccount/ :
    source source_setup_Meso.sh
    then follow the instructions in the file create MyVirtualEnvironment in order to create a virtual environment for Tensorflow
  • in /home/myaccount/(path_to_the_directory_of_my_code) :
    copy, modify the shell script myjob Meso and put it in this directory
    sbatch -p volta --gres=gpu:1 ./myjob_Meso.sh (volta or pascal or kepler)
  • Be careful ! You don't need to specify the number of CPUs you want. It will be automatically determined by the amount of RAM you asked for. Therefore, the actual number of counted hours will be : number of CPUs * number of hours of your simulation.

Workshops / Conferences¶

• 07-11/06/2021 : The SF2A week, workshop "Machine Learning for the study of galaxies and cosmology"
  https://www.carbonfreeconf.com/website/141/home
• 25-29/01/2021 : Mathématiques, traitement du signal et apprentissage​
  • programme https://conferences.cirm-math.fr/2472.html
  • supports https://www.i2m.univ-amu.fr/seminaires_signal_apprentissage/Conf/Jan2021/index.php
  • vidéos : https://library.cirm-math.fr/ListRecord.htm?list=request&table=3&NumReq=115&cluster_1=2472
• 07-08/07/2020 : Programmer et déployer votre IA (objectifs et vidéos http://devlog.cnrs.fr/jdev2020/t8)

Initial meetings (in French)¶

• Visioconférence du 09/10/2020
• 21/10/2020 : Présentation du pôle MLDL au CS du LAM [ CeSAM__2020-v3.pdf ] [ CeSAM__2020-v3.pptx ]