• Table of contents
• Wiki of the Machine Learning / Deep Learning Pole
  • Seminars
  • Reference publications
  • Books (Top 6) & Code Examples
  • Frameworks to start & to document in ML/DL
  • Next training workshops or conferences
  • Previous conferences on ML/DL with astrophysical topics
  • Previous training workshops
  • Online courses
  • Available GPUs computing resources for LAM staff
  • Initial meetings (in French)

Wiki of the Machine Learning / Deep Learning Pole¶

INFORMATION WEB PAGE for the ML/DL Pole at CeSAM
This is a selection of some references that may be useful to start or consolidate your knowledge. This information is of course not exhaustive and any suggestions for additions are welcome. If you have any questions, please send an email to morgan.gray@lam.fr

Seminars¶

For the presentations, the slides can be retrieved with the link; the videos can be viewed on https://seminars.lam.fr/#MLDL

• 04/05/2021 : Raoul Canameras (MPA-Garching) " Finding and modeling strong gravitational lenses with deep neural networks " (210504 Canameras)
• 30/03/2021 : Alexandre Boucaud & Hubert Bretonnière (LAC) " FlowVAE: taking control of galaxy image simulations with deep generative networks "
• 09/03/2021 : Laurent Risser (IMT) " Explainability techniques for black-box decision rules in Machine Learning " (210309 Risser)
• 11/02/2021 : Sidonie Lefebvre (ONERA/DOTA) " Generative Adversarial Networks (GANs) : concept and application to cloudy sky images synthesis " (210211 Lefevbre)
• 01/02/2021 : Nicolas Audebert (CNAM) " Hyperspectral remote sensing data analysis using Deep Learning " (210201 Audebert)
• 18/01/2021 : François-Xavier Dupé (LIS/QARMA) " How Machine Learning can help to automate processing tasks ? An example with image denoising " (210118 Dupe)
• 11/01/2021 : Julien Wojak (Institut Fresnel) " Deep Learning : focus on auto-encoder as a pre-processing step for classification " (210111 Wojak)

Reference publications¶

• General comments on :
  • Machine Learning : Baron(19) MLinAstronomyOverview & Mehta MLforPhysicists
  • Deep Learning : Hadji(18) CNN-Overview
• Some classical Neural Network architectures :
  • VGGNet : Simonyan VGGNet ; ResNet : He(15) ResNet & He(16) ResNet
  • Inception : Szegedy(14) Inception, Szegedy(15) RethinkingInception & Szegedy(16) InceptionV4 InceptionResNet
  • Xception : Chollet Xception
  • Generative Adversarial Networks : Goodfellow GenerativeAdversarialNets
  • Adversarial Auto Encoders : Makhzani AdversarialAutoencoders
• Practical advice (methodology, specific aspects) :
  • Hyperparameters utilization : Bengio(12) PracticalRecommendations & Smith(18) Hyperparameters
  • Optimizers : OptimizerAlgorithms ; Dropout : Dropout ; Batchnormalisation : BatchNormalization
• Publications (research in ML/DL) :
  • Papers with Code: https://paperswithcode.com/

Books (Top 6) & Code Examples¶

• Hands-On Machine Learning with Scikit-Learn, Keras & Tensorflow by Aurélien Géron (O'Reilley editor, second edition)
  • Jupyter notebooks for the code samples : https://github.com/ageron/handson-ml2
• Deep Learning with Python by François Chollet (Manning editor) : a good presentation of Keras to make Deep Learning
  • Jupyter notebooks for the code samples : https://github.com/fchollet/deep-learning-with-python-notebooks
  • Examples of codes with different problems & neural networks : https://keras.io/examples/ ; https://keras.io/api/applications/
• Deep Learning by Goodfellow, Bengio, Courville (MIT press book editor) : a more theoretical book, free access per chapter https://www.deeplearningbook.org/
• Pattern Recognition & Machine Learning by Christopher M. Bishop (Springer Editor)
• Statistics, Data Mining and Machine Learning in Astronomy by Ivezic, Connolly, Vanderplas & Gray (Princeton Press)
  • HTML documentation : https://www.astroml.org/ ; Source-code repository: https://github.com/astroML/astroML
• The Elements of Statistical Learning by Hastie, Tibshirani & Friedmman (Springer Editor) : free to download https://web.stanford.edu/~hastie/ElemStatLearn/

Frameworks to start & to document in ML/DL¶

• ScikitLearn : https://scikit-learn.org/stable/user_guide.html
• Keras : https://keras.io/api/ ou https://keras.io/getting_started/intro_to_keras_for_engineers/
• Tensorflow : https://www.tensorflow.org/api_docs/python/tf
• Pytorch : https://pytorch.org/get-started/locally/ ; https://pytorch.org/tutorials/
• MLflow, an Open Machine Learning Platform : https://mlflow.org/

Next training workshops or conferences¶

• 29/11-10/12/2021 : Ecole Thématique AstroInformatique https://astroinfo2021.sciencesconf.org/

Previous conferences on ML/DL with astrophysical topics¶

• Debating the potential of ML in astronomical surveys : https://ml-iap2021.sciencesconf.org/browse/session

Previous training workshops¶

• FIDLE 2021 (in French) : Introduction au Deep Learning (un très bon cours !)
  • Programme : http://bit.ly/fidle-a-distance-programme
  • Readme : https://gricad-gitlab.univ-grenoble-alpes.fr/talks/fidle/-/blob/master/README.md
  • supports : https://gricad-gitlab.univ-grenoble-alpes.fr/talks/fidle
  • videos : https://www.youtube.com/channel/UC4Sukzudhbwr6fs10cXrJsQ
• CIRM 2021 (in English) : Mathematics, Signal Processing and Learning ​ (more theoretical courses with a mathematical view & practical sessions) : https://conferences.cirm-math.fr/2472.html
  • slides : https://www.i2m.univ-amu.fr/seminaires_signal_apprentissage/Conf/Jan2021/index.php
  • videos : https://library.cirm-math.fr/ListRecord.htm?list=request&table=3&NumReq=115&cluster_1=2472
• Ecole CNRS 2020 (in French) : ANF Machine Learning pour Informaticiens, bases et enjeux du Machine Learning et du Deep Learning (une bonne formation, ne pas hésiter à aller "piocher" dans les différents notebooks suivant les centres d'intérêts)
  supports : https://gitlab.in2p3.fr/ri3/ecole-info/2020/anf-machine-learning/-/tree/master/notebooks
• JDEV 2020 (in French) :
  • Python, préparation des données avec Panda & Apprentissage Automatique avec Scikitlearn : http://devlog.cnrs.fr/jdev2020/t8.a05
    supports : https://gitlab.in2p3.fr/aboucaud/atelier-jdev-2020
  • Fondamentaux Apprentissage Automatique avec Python & ScikitLearn : http://devlog.cnrs.fr/jdev2017/t8.a04
    supports : http://laurent.risser.free.fr/TMP_SHARE/JDEV2020/
  • Introduction Keras & Tensorflow : http://devlog.cnrs.fr/jdev2020/t8.ap04
    supports : https://gricad-gitlab.univ-grenoble-alpes.fr/talks/fidle/
  • Introduction Deep Learning & PyTorch : http://devlog.cnrs.fr/jdev2020/t8.ap01
    supports : http://laurent.risser.free.fr/TMP_SHARE/JDEV2020_T8_AP01/
  • Programmer et déployer votre IA (objectifs et vidéos http://devlog.cnrs.fr/jdev2020/t8)

Online courses¶

• CNAM (in French) :
  • RCP208 : Reconnaissance des formes et méthodes neuronales (http://cedric.cnam.fr/vertigo/Cours/ml/)
  • RCP209 : Apprentissage, réseaux de neurones et modèles graphiques (http://cedric.cnam.fr/vertigo/Cours/ml2/)
  • Cours de Nicolas Thome : slides Formation CNAM ; videos : send an email request
• ML/DL Mastery (in English) : https://machinelearningmastery.com/start-here/
• KAGGLE (in English) : https://www.kaggle.com/learn/overview ; https://www.kaggle.com/learn/intro-to-machine-learning ;
  https://www.kaggle.com/learn/intro-to-deep-learning ; https://www.kaggle.com/learn/computer-vision
• STANFORD University (in English) :
  • CNN for Visual Recognition : https://cs231n.github.io/ ou http://cs231n.stanford.edu/syllabus.html
  • COURSERA : slides Formation COURSERA ; videos https://www.coursera.org/learn/machine-learning#syllabus

Available GPUs computing resources for LAM staff¶

• LAM cluster (responsible JC Lambert) : 7 GPUs RTX2080 + 3 GPUs RTX6000 + 5 GPUs A40
  https://projets.lam.fr/projects/cluster-de-calcul-du-lam/wiki#GPU-partition
• Aix-Marseille University Mesocentre cluster : several partitions (for the volta partition: more than 20 GPUs V100)
  https://mesocentre.univ-amu.fr/appel-a-projets/
• How to submit a job on the Mesocentre cluster :
  • in /home/myaccount/ :
    copy the shell script source setup Meso and put it in this directory
  • in /home/myaccount/ :
    source source_setup_Meso.sh
    then follow the instructions in the file create MyVirtualEnvironment in order to create a virtual environment for Tensorflow
  • in /home/myaccount/(path_to_the_directory_of_my_code) :
    copy, modify the shell script myjob Meso and put it in this directory
    sbatch -p volta --gres=gpu:1 ./myjob_Meso.sh (volta or pascal or kepler)
  • Be careful ! You don't need to specify the number of CPUs you want. It will be automatically determined by the amount of RAM you asked for. Therefore, the actual number of counted hours will be : number of CPUs * number of hours of your simulation.

Initial meetings (in French)¶

• Visioconférence du 09/10/2020
• 21/10/2020 : Présentation du pôle MLDL au CS du LAM [ CeSAM__2020-v3.pdf ] [ CeSAM__2020-v3.pptx ]