# Instruction for the courses: ### Introduction to Unreal Engine ### Virtual Worlds for Machine Learning (VWML) ## General Information The introduction course will take place as an online event on 3 successive Fridays. The link to the streaming platform will be provided to the registrants only. The Virtual Worlds for Machine Learning course will take place on _two consecutive days_, March 21st and March 22nd. The link to the streaming platform will be provided to the registrants only. This year, there will not be an option to attend in person. ### Support: Write an e-mail to Dirk Helmrich and Jonathan Windgassen: d.helmrich@fz-juelich.de j.windgassen@fz-juelich.de ### Roadmap and speakers: - Introduction to Unreal Engine - Introduction to Unreal Engine (UE) - Dirk Helmrich - Programming with UE-Blueprints - Jonathen Windgassen - Programming with C++ in UE - Dirk Helmrich - Virtual Worlds for Machine Learning - Dirk Helmrich - Scalability, Generalization, Domain Visualization - Specific application case: plant science - Introduction to Training concepts, visualization - Digital Twin modelling of experiments - Building an AI/ML pipeline from WebRTC - Preparing the frameworks - Parsing and using data - Best practices ### Prerequisites: - General programming skills - Dedicated graphics card preferred, integrated graphics only sufficient for low workloads - Installed and running version of Unreal Engine 5.3 - For participants of the UE/ML course, we refer you to free links on deep learning concepts - The MIT introduction to Deep Learning Course (http://introtodeeplearning.com/) - The Machine Learning course and Deep Learning specialization by Andrew Ng et al. at Stanford (https://cs230.stanford.edu/) and on Coursera (www.coursera.org) - The notebook-based courses of fast.ai (www.fast.ai) and of Master Datascience Paris Saclay (https://github.com/m2dsupsdlclass/lectures-labs) - Deep Learning, MIT Press book, Ian Goodfellow, YOshua Bengio, Aaron Courville (https://www.deeplearningbook.org/) - The UE/ML course should be visited with previous knowledge of Unreal Engine. While it provides some insight into how Unreal Engine work, it is not to be understood as an introduction. ## Course Material - The GitHub project can be found here: https://github.com/dhelmrich/UnrealCourse - Videos will be uploaded and shared after the course has concluded ## Preparation ### We are investigating whether any install is necessary before the June course. To be well prepared, we ask you to install the Unreal Engine 5.3.* and a C++ Development IDE such as Visual Studio 2022 upfront. To do so, please proceed as follows: 1) Create an "Epic Games" account and install the "Epic Games Launcher": http://epicgames.com/store 2) In the "Epic Games Launcher" under the menu item Library install the Unreal Engine in the Version 5.3.*. Attention, you will need about 25GB of free space, so you might want to adjust the installation path. 3) Install the "Visual Studio 2022 Community Edition" and unlock it with a Microsoft account if needed. This version is also for free: http://visualstudio.microsoft.com/downloads Participants can also choose to install Visual Studio Code, which is free and available for download on multiple OS: https://code.visualstudio.com/ For proper installation and setup, please refer to the following document. The "Unreal Engine Installer" under "Installation Details" can now be omitted. This has already been done in step 1: https://docs.unrealengine.com/en-US/ProductionPipelines/DevelopmentSetup/VisualStudioSetup/index.html. The Unreal Engine offers support for a variety of IDEs, such as Rider, but we encourage participants to follow the default guidelines as closely as possible. If you are encountering any problem, do not hesitate to contact us. However, please understand that daily workloads or the specifics of your problems might impact response time. ## Registration is handled via the conference management system Indico. Link to the registration for the introduction course: https://indico3-jsc.fz-juelich.de/unreal/ ## Attendance of the UE-ML course Registered PhD Students who have full attendance will receive 3CP for their studies. Students that are not PhenoRob students can submit the FZJ certificate to their graduate school, but we take no responsibility on whether that gets accepted or not. ## Troubleshooting ### 1. Start Unreal via the Epic Games launcher for the first time, to install prerequisites This includes DirectX and Redistributables that need to be installed for the runtime environment to work. We do not know whether this is related to any windows errors people are getting, but make sure to also do this. ### 2. If the project still does not load, create a new project There is only the skysphere in the template project, and this can be easily reproducible. You can copy the `.uasset` files over and import them into the new project. This should circumvent some processes that might lead to issues on participants PCs. 1. Create a new Project at a location where you have some disk space (~4GB) 2. Set the Starter Content to activated and the project type to C++ 3. Open the project 4. Under "create", create a new map and choose the blank layout ## Support Information ### SSH keys You will have received instructions to generate an SSH key. If everything is going to plan, however, we will be using Jupyter-JSC for the cluster parts of the course. As such, I would recommend you to not take action at this point. ### Software for the VWML course I will be using the following software during teaching. You are not strictly required to install them; in this course, I would like to keep a lively interaction. However, I will briefly list the software that I will definitely use to show-case systems. I will mark with a star* software that is already available on the cluster. We will work both locally and remotely during the course. Details about potential cluster access during that time are still pending organization. For the Unreal Engine, I also refer to our description we also use for the introduction course above. - Unreal Engine - An IDE such as Visual Studio - OpenCV/GStreamer* - Tensorflow* - Synavis and SynavisUE - Paraview* ### Cluster Access I would like you to make an account at Judoor: https://judoor.fz-juelich.de/register Afterwards, I would like you to sign up for the following project: **[Link Pending]** ### Notes Again, if you require assistance, please reach out. The course will cover a number of techniques, as so far, the systems are considered current research. We will additionally touch upon established methods, such as UnrealCV, but not in as much depth. For the Synavis Project, we might add a discussion round at the end if that is feasible with our media equipment.