• Table of contents
• Wiki of the Machine Learning / Deep Learning Pole
  • Reference publications
  • Books & Code Examples
  • Frameworks to start & to document in ML/DL
  • Online courses
  • Next conferences or training workshops
  • Previous conferences on ML/DL with astrophysical topics
  • Previous training workshops
  • Seminars
  • ML/DL pole staff & project implications
  • Available GPUs computing resources for LAM staff

Wiki of the Machine Learning / Deep Learning Pole¶

INFORMATION WEB PAGE for the ML/DL Pole at CeSAM
This is a selection of some references that may be useful to start or consolidate your knowledge. This information is of course not exhaustive and any suggestions for additions are welcome. If you have any questions, please send an email to morgan.gray@lam.fr

Reference publications¶

• General comments on :
  • Machine Learning : Baron(19) MLinAstronomyOverview & Mehta MLforPhysicists
  • A glossary that defines general ML terms : https://developers.google.com/machine-learning/glossary#a
  • Deep Learning : Hadji(18) CNN-Overview
• Some classical Neural Network architectures :
  • Simonyan VGGNet - He(15) ResNet - He(16) ResNet
  • Szegedy(14) Inception - Szegedy(15) RethinkingInception - Szegedy(16) InceptionV4 InceptionResNet - Chollet Xception
  • Makhzani AdversarialAutoencoders - Goodfellow GenerativeAdversarialNets
• Practical advice (methodology, specific aspects) :
  • Bengio(12) PracticalRecommendations - Smith(18) Hyperparameters - OptimizerAlgorithms - Dropout - BatchNormalization

Books & Code Examples¶

• Hands-On Machine Learning with Scikit-Learn, Keras & Tensorflow by Aurélien Géron (O'Reilley editor, second edition)
  • Jupyter notebooks for the code samples : https://github.com/ageron/handson-ml2
• Deep Learning with Python by François Chollet (Manning editor) : a good presentation of Keras to make Deep Learning
  • Jupyter notebooks for the code samples : https://github.com/fchollet/deep-learning-with-python-notebooks
  • Examples of codes with several issues & neural networks : https://keras.io/examples/ ; https://keras.io/api/applications/
• Deep Learning with PyTorch by Eli Stevens, Luca Antiga, and Thomas Viehmann (Manning editor)
• Deep Learning by Goodfellow, Bengio, Courville (MIT press book editor) : a more theoretical book
  • free access per chapter : https://www.deeplearningbook.org/
• Probabilistic Deep Learning with Python, Keras and Tensorflow Probability by Durr, Sic, Murina (Manning editor)
• Bayesians methods for hackers, Probabilistic Programming and Bayesian Inference by Cameron Davidson-Pilon (Addison-Wesley Data & Analytics editor)
  • free access per chapter : https://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/
• Pattern Recognition & Machine Learning by Christopher M. Bishop (Springer Editor)
  • free to download : http://mng.bz/oPWZ
• Statistics, Data Mining and Machine Learning in Astronomy by Ivezic, Connolly, Vanderplas & Gray (Princeton Press)
  • HTML documentation : https://www.astroml.org/ ; Source-code repository: https://github.com/astroML/astroML
• The Elements of Statistical Learning by Hastie, Tibshirani & Friedmman (Springer Editor)
  • free to download : https://web.stanford.edu/~hastie/ElemStatLearn/

Frameworks to start & to document in ML/DL¶

• ScikitLearn : https://scikit-learn.org/stable/user_guide.html
• Keras : https://keras.io/getting_started/ ; https://keras.io/api/ ; https://keras.io/guides/
• Tensorflow : https://www.tensorflow.org/api_docs/python/tf
• Pytorch : https://pytorch.org/get-started/locally/ ; https://pytorch.org/tutorials/
• MLflow, an Open Machine Learning Platform : https://mlflow.org/

Online courses¶

• JRES 2022 (in French) : Le Deep Learning en 100 minutes !
  • https://replay.jres.org/w/kzX7FHoPJSazgfxrJEwMWK
• FIDLE 21/22 (in French) : Introduction au Deep Learning (un très bon cours !)
  • Programme : https://gricad-gitlab.univ-grenoble-alpes.fr/talks/fidle/-/wikis/Fidle%20%C3%A0%20distance/Programme
  • videos : https://www.youtube.com/c/CNRSFormationFIDLE
  • supports : https://gricad-gitlab.univ-grenoble-alpes.fr/talks/fidle
• CNAM (in French) :
  • Réseaux de neurones : slides Formation CNAM ; videos : send an email request
  • Modélisation descriptive et introduction aux réseaux de neurones http://cedric.cnam.fr/vertigo/Cours/ml/
  • Modélisation décisionnelle et apprentissage profond http://cedric.cnam.fr/vertigo/Cours/ml2/
• STANFORD University (in English) :
  • Machine Learning : https://www.coursera.org/learn/machine-learning#syllabus ; slides Formation COURSERA
  • CNN for Visual Recognition : https://cs231n.github.io/
• ML/DL Mastery (in English) : https://machinelearningmastery.com/start-here/

Next conferences or training workshops¶

• European Astronomical Society annual meeting, session Machine Learning (Valencia, Spain from 27 June to 1 July 2022)
  • A giant leap towards space discovery in the era of peta and exabyte scale surveys : https://eas.unige.ch/EAS_meeting/session.jsp?id=S11
  • Critical Challenges for Machine Learning in Astronomy : https://eas.unige.ch/EAS_meeting/session.jsp?id=SS24
  • Towards the SKA Observatory: Artificial Intelligence in radio astronomy : https://eas.unige.ch/EAS_meeting/session.jsp?id=SS23

Previous conferences on ML/DL with astrophysical topics¶

• Bayesian Deep Learning for Cosmology & Time Domain Astrophysics : https://astrodeep.net/workshop2022/
• ESO/ESA SciOps workshop on Artificial Intelligence for Science and Operations in Astronomy : https://www.eso.org/sci/meetings/2022/SCIOPS2022/program.html
• Ecole CNRS 2021 (in French) : Ecole Thématique AstroInformatique : https://astroinfo2021.sciencesconf.org/program
• Debating the potential of ML in astronomical surveys : https://ml-iap2021.sciencesconf.org/browse/session.
• Ecole CNRS 2020 (in French) : ANF Machine Learning pour Informaticiens, bases et enjeux du Machine Learning et du Deep Learning : https://gitlab.in2p3.fr/ri3/ecole-info/2020/anf-machine-learning/-/tree/master/notebooks
• Bayesian Deep Learning for Cosmology & Gravitational waves : https://astrodeep.net/workshop2020/#schedule

Previous training workshops¶

• CIRM 2021 (in English) : Mathematics, Signal Processing and Learning ​ (more theoretical courses with a mathematical view & practical sessions)
  • https://conferences.cirm-math.fr/2472.html
  • slides : https://www.i2m.univ-amu.fr/seminaires_signal_apprentissage/Conf/Jan2021/index.php
  • videos : https://library.cirm-math.fr/ListRecord.htm?list=request&table=3&NumReq=115&cluster_1=2472
• JDEV 2020 (in French) :
  • Python, préparation des données avec Panda & Apprentissage Automatique avec Scikitlearn : http://devlog.cnrs.fr/jdev2020/t8.a05
    supports : https://gitlab.in2p3.fr/aboucaud/atelier-jdev-2020
  • Fondamentaux Apprentissage Automatique avec Python & ScikitLearn : http://devlog.cnrs.fr/jdev2017/t8.a04
    supports : http://laurent.risser.free.fr/TMP_SHARE/JDEV2020/
  • Introduction Keras & Tensorflow : http://devlog.cnrs.fr/jdev2020/t8.ap04
    supports : https://gricad-gitlab.univ-grenoble-alpes.fr/talks/fidle/
  • Introduction Deep Learning & PyTorch : http://devlog.cnrs.fr/jdev2020/t8.ap01
    supports : http://laurent.risser.free.fr/TMP_SHARE/JDEV2020_T8_AP01/
  • Programmer et déployer votre IA (objectifs et vidéos http://devlog.cnrs.fr/jdev2020/t8)

Seminars¶

For the presentations, the slides can be retrieved with the link; the videos can be viewed on https://seminars.lam.fr/#MLDL

• 25/03/2022 : François Lanusse (CosmoStat) " Probabilistic Deep Learning for Weak Lensing : from Mass-Mapping to Cosmological Parameter Inference " (https://eiffl.github.io/LAM2022/)
• 04/05/2021 : Raoul Canameras (MPA-Garching) " Finding and modeling strong gravitational lenses with deep neural networks " (210504 Canameras)
• 30/03/2021 : Alexandre Boucaud & Hubert Bretonnière (LAC) " FlowVAE: taking control of galaxy image simulations with deep generative networks "
• 09/03/2021 : Laurent Risser (IMT) " Explainability techniques for black-box decision rules in Machine Learning " (210309 Risser)
• 11/02/2021 : Sidonie Lefebvre (ONERA/DOTA) " Generative Adversarial Networks (GANs) : concept and application to cloudy sky images synthesis " (210211 Lefevbre)
• 01/02/2021 : Nicolas Audebert (CNAM) " Hyperspectral remote sensing data analysis using Deep Learning " (210201 Audebert)
• 18/01/2021 : François-Xavier Dupé (LIS/QARMA) " How Machine Learning can help to automate processing tasks ? An example with image denoising " (210118 Dupe)
• 11/01/2021 : Julien Wojak (Institut Fresnel) " Deep Learning : focus on auto-encoder as a pre-processing step for classification " (210111 Wojak)

ML/DL pole staff & project implications¶

• Proposal for the creation of the ML/DL pole : Visioconférence du 09/10/2020
• Presentation of the ML/DL pole to the LAM scientific council : CeSAM__2020-v3.pdf
• Presentation of the ML/DL pole at the OSU-AI scientific day (20/04/2022): Gray JourneeIA-OSU
• Poster of the ML/DL pole at the LAM/ONERA/FRESNEL day (16/06/2022): Gray PosterPoleMLDL-A4
• Staff (FTE) : Morgan Gray (100%) ; Simon Conseil (50%) ; Jean-Charles Meunier (25%) ; Christian Surace (5%) ; Jean-Charles Lambert (5%) ; Didier Vibert (5%) ; Charleston Chauvet (40%)
• Projects :
  • ANR APPLY : https://sites.google.com/view/anr-apply/home
  • EUCLID : https://sci.esa.int/web/euclid (Ground Segment : https://www.euclid-ec.org/?page_id=2625)
  • BigSF : https://people.lam.fr/zavagno.annie/big_data_and_machine_learning.html
  • AZIMOV :

Available GPUs computing resources for LAM staff¶

• LAM cluster (responsible JC Lambert) : 7 GPUs RTX2080 + 3 GPUs RTX6000 + 8 GPUs A40
  https://projets.lam.fr/projects/cluster-de-calcul-du-lam/wiki#GPU-partition (see also https://shorturl.at/uBD56 for the LAM cluster)
• Aix-Marseille University Mesocentre cluster : several partitions (for the volta partition: more than 20 GPUs V100)
  https://mesocentre.univ-amu.fr/appel-a-projets/
• How to submit a job on the Mesocentre cluster :
  • in /home/myaccount/ :
    copy the shell script source setup Meso and put it in this directory
  • in /home/myaccount/ :
    source source_setup_Meso.sh
    then follow the instructions in the file create MyVirtualEnvironment in order to create a virtual environment for Tensorflow
  • in /home/myaccount/(path_to_the_directory_of_my_code) :
    copy, modify the shell script myjob Meso and put it in this directory
    sbatch -p volta --gres=gpu:1 ./myjob_Meso.sh (volta or pascal or kepler)
  • Be careful ! You don't need to specify the number of CPUs you want. It will be automatically determined by the amount of RAM you asked for. Therefore, the actual number of counted hours will be : number of CPUs * number of hours of your simulation.