Reinforcement Learning Course
Dr. Rohban and Mr. Hasani
Spring 2023
Session 23

Reinforcement Learning Course
Dr. Rohban and Mr. Hasani
Spring 2023
Session 25

Reinforcement Learning Course
Dr. Rohban and Mr. Hasani
Spring 2023
Session 26

سلام به همه‌ی دوستان
عزیزانی که علاقه‌مند به پژوهش در حوزه یادگیری ماشین اعتمادپذیر در آزمایشگاه RIML هستند، لطفا رزومه‌ و مدت زمانی که براشون مقدور هست به پژوهش اختصاص بدهند را به ایمیل زیر ارسال کنند. نهایتاََ یک تیم از دوستان علاقه‌مند تشکیل می‌شود و یک فرآیند آموزش و در ادامه تعریف صورت مسئله برای پژوهش صورت خواهد گرفت که خروجی نهایی در قالب مقاله برای کنفرانس‌های مرتبط ML ارسال خواهد شد. بحث توصیه‌نامه برای دوستان حاضر در پروژه از سمت اساتید در نظر گرفته می‌شود.
کارهای مشابه آزمایشگاه در این حوزه را در این مقاله و این لینک می‌توانید بررسی بفرمایید.
hsirm97@gmail.com

ویدئوهای درس پردازش هوشمند تصاویر زیست-پزشکی، دکتر رهبان، پائیز ۱۴۰۲:
https://www.aparat.com/playlist/7606670

ویدئوهای درس یادگیری تقویتی، دکتر رهبان، بهار ۱۴۰۳:
https://www.aparat.com/playlist/9319081

#اخبار_پژوهشی_آزمایشگاه

چاپ مقاله
RODEO: Robust Outlier Detection via Exposing Adaptive Outliers
در کنفراس ICML2024 تحت نظارت آقای دکتر رهبان

https://www.linkedin.com/posts/mohammad-hossein-rohban-75567677_we-are-pleased-that-our-most-recent-paper-activity-7191832868602478592-TipI?utm_source=share&utm_medium=member_android

#موقعیت_پژوهشی

سلام به همه‌ی دوستان.

قابل توجه علاقه‌مندان به پژوهش در حوزه‌ی کشف دارو (پیشبینی خواص مولکولی) با روش‌های مبتنی بر شبکه‌های عصبی عمیق، پروژه‌ای با این عنوان در آزمایشگاه RIML تعریف شده است.

پیش‌نیاز:
- تسلط کافی بر مباحث یادگیری عمیق
- آشنایی کافی با فریم‌ورک‌های یادگیری عمیق نظیر پایتورچ

موارد امتیازی:
- آشنایی با شبکه‌های عصبی گرافی
- آشنایی با مفاهیم شیمی و خواص مولکولی
- آشنایی با ابزارهایی مانند RdKit

مزایا:
- ریسرچ زیر نظر دکتر رهبان در آزمایشگاه RIML
- یادگیری مفاهیم پیشرفته در شبکه‌های عصبی گرافی و پیاده‌سازی آن‌‌ها
- ارسال پیپر در کنفرانس‌ یا ژورنال مرتبط

برای آشنایی بیشتر با این حوزه‌ی ریسرچ می‌توانید مقالات زیر را بررسی نمایید:
https://www.nature.com/articles/s41467-023-41948-6
https://arxiv.org/abs/2206.00133
https://arxiv.org/abs/2106.07971
https://arxiv.org/abs/2106.06130

در صورت تمایل، می‌توانید رزومه‌ی خود را به آیدی زیر ارسال کنید:
@aminreza_sefid

Adversarial robust learning and its generalization issues


This is a research project in the group of Dr. Rohban (RIML lab) from Sharif University of Technology

Project description:
Despite deep neural networks impressive success in many real-world problems, their instability under test-time adversarial noises is the major issue against their use in safety-critical applications. Therefore, the problem of learning robust deep networks (not only accurate on original samples, but also accurate on adversarially perturbed ones) has become an active area of research.
Training the model based on the adversarial samples in each mini-batch, which is known as “Adversarial training” (AT), has been empirically established as a general and effective approach to remedy this issue. However, real challenges in practice and also theoretical aspects have remained. Especially, we face some critical generalization issues in this new learning paradigm including the larger generalization gap between test and train data in comparison with standard training or the specific phenomenon called catastrophic overfitting. Achieving a better understanding of this topic can be a good help to provide more robust models.

In this project, we aim to analyze generalization in robust learning in a more comprehensive, deep, and detailed way. The project has both theoretical and practical aspects; So having interest, capability, and perseverance in both aspects is needed.

Estimated time for the project is 6 months although it may change depending on the progress and results of the project.

For more information, you can read the following paper:
Zerograd: Costless conscious remedies for catastrophic overfitting in the FGSM adversarial training

Requirements:
- Familiarity with linear algebra fundamentals
- Familiarity with statistics and probability
- Familiarity with ML and deep learning fundamentals
- Hands-on experience in ML and deep learning
- Hands-on experience with PyTorch framework
- Dedicating considerable time and consistency to the project 
- Enthusiasm to learn and tackle research problems

Preferred qualifications:
** Familiarity with Jax framework 
* Familiarity with adversarial robustness


To apply for the position, please read the suggested paper and send your resume as well as your research interests to z.golgooni@gmail.com

We would be happy to answer any questions you may have through the above email.

#open_position
#research_application

با سلام خدمت همه دوستان
رویداد رایان با هدایت دکتر رهبان به زودی برگزار خواهد شد. تیم علمی رایان به دنبال گسترش و همکاری با دانشجویان گرامی علاقه‌مند می‌باشد. مسولیت‌های اصلی اعضای علمی شامل هندل کردن دوره آموزشی (e.g. تولید محتوا/تمرین) و همچنین طراحی چالش (e.g. پیاده سازی ایده ها و شرکت در طراحی مسئله) می باشد. مزایای عضویت در تیم علمی شامل دریافت حقوق و ریکام (در صورت تایید هد تیم علمی) می‌باشد. شما عزیزان می‌توانید با پرکردن این فرم علاقه‌مندی خود را برای عضویت در تیم علمی اعلام بفرمایید. شایان ذکر است که از اعضای تیم انتظار می‌رود که هفته‌ای حداقل ۱۵ ساعت زمان به رویداد اختصاص بدهند. مهلت پرکردن این فرم تا فردا خواهد بود.

#اخبار_پژوهشی_آزمایشگاه

مقالات برتر چاپ شده از آغاز سال ۲۰۲۳ تحت نظارت آقای دکتر رهبان

Fake It Until You Make It: Towards Accurate Near-Distribution Novelty Detection
آقای حسین میرزایی دانشجوی کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در کنفرانس ICLR

Lagrangian objective function leads to improved unforeseen attack generalization
آقای محمد عزیزملایری دانشجوی دکترا آزمایشگاه RIML - چاپ شده در ژورنال Machine Learning

Compositions and methods for treating proliferative diseases
US Patent App.

Zerograd: Costless conscious remedies for catastrophic overfitting in the fgsm adversarial training
خانم زینب گلگونی دانشجوی دکترا آزمایشگاه RIML - چاپ شده در ژورنال Intelligent Systems with Applications

A deep learning framework to scale linear facial measurements to actual size using horizontal visible iris diameter: a study on an Iranian population
آقای دکتر حسین محمدرحیمی محقق در آزمایشگاه RIML - چاپ شده در ژورنال Scientific Reports

Weakly-Supervised Drug Efficiency Estimation with Confidence Score: Application to COVID-19 Drug Discovery
خانم نهال میرزایی و آقای محمد ولی‌ثانیان دانشجویان کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در کنفرانس MICCAI

Forecasting influenza hemagglutinin mutations through the lens of anomaly detection
آقای محمدرضا صالحی دانشجوی اسبق آزمایشگاه RIML - چاپ شده در ژورنال Scientific Reports

Borderless azerbaijani processing: Linguistic resources and a transformer-based approach for azerbaijani transliteration
خانم ریحانه زهرابی دانشجوی کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در کنفرانس ACL - تحت نظر دکتر بیگی، دکتر عسگری و دکتر رهبان

Examination of lemon bruising using different CNN-based classifiers and local spectral-spatial hyperspectral imaging
آقای دکتر سجاد سبزی پسادکترا آزمایشگاه RIML - چاپ شده در ژورنال Algorithms

A Robust Heterogeneous Offloading Setup Using Adversarial Training
آقای مهدی امیری دانشجوی کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در ژورنال IEEE Transactions on Mobile Computing - تحت نظر دکتر رهبان و دکتر حسابی

Universal Novelty Detection Through Adaptive Contrastive Learning
آقای حسین میرزایی و آقای مجتبی نافذ دانشجویان کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در کنفرانس CVPR

Killing It With Zero-Shot: Adversarially Robust Novelty Detection
آقای حسین میرزایی دانشجوی کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در کنفرانس IEEE ICASSP

User Voices, Platform Choices: Social Media Policy Puzzle with Decentralization Salt
جمعی از دانشجویان کارشناسی - چاپ شده در کنفرانس CHI

Comparison of 2D and 3D convolutional neural networks in hyperspectral image analysis of fruits applied to orange bruise detection
آقای دکتر سجاد سبزی پسادکترا و خانم ریحانه زهرابی دانشجوی کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در Journal of Food Science

RODEO: Robust Outlier Detection via Exposing Adaptive Outliers
آقای حسین میرزایی و آقای مجتبی نافذ دانشجویان کارشناسی‌ارشد آزمایشگاه RIML - چاپ شده در کنفرانس ICML

Virtual screening for small-molecule pathway regulators by image-profile matching
آقای دکتر رهبان و خانم دکتر Anne E. Carpenter - چاپ شده در ژورنال Cell systems

Khayyam Challenge (PersianMMLU): Is Your LLM Truly Wise to The Persian Language?
Coming Soon! :)

و ده‌ها مقاله دیگر که در Google Scholar دکتر رهبان می‌توانید مشاهده کنید.

تبریک خدمت تمامی اعضای آزمایشگاه به دلیل تلاش‌‌، کوشش و پژوهش در جهت رفع مشکلات جامعه و کشور و چاپ مقالات در برترین کنفرانس‌ها و مجلات AI

Project Description:
This project is a collaborative effort between Dr. Rohban, Dr. Soleymani, and Dr. Asgari. Together, we aim to push the boundaries of language model evaluation for the Persian language. In this project, our primary goal is to benchmark and develop innovative methods for evaluating language models on the Persian language both robustly and comprehensively. Our approach will encompass both static and dynamic assessments to ensure thorough analysis. This initiative seeks to advance the field by addressing unique challenges posed by Persian language processing.

For more in-depth insights, please refer to the following papers:

"Khayyam Challenge (PersianMMLU): Is Your LLM Truly Wise to The Persian Language?"

Requirments:
Familiarity with LLM Concepts: Understanding the fundamentals and advancements in large language models.
Deep Learning Expertise: Practical knowledge and experience in deep learning techniques.
PyTorch Proficiency: Hands-on experience with the PyTorch framework is essential.
Commitment: Ability to dedicate significant time and maintain consistency throughout the project.


To apply for this position, please read the suggested papers and send your resume along with a brief summary of your research interests to mvs2667@gmail.com. We are eager to hear from motivated individuals who are passionate about advancing language model evaluation.

For any inquiries, feel free to reach out to us via the above email.

#open_position
#research_application

#اخبار_پژوهشی_آزمایشگاه

چاپ ۲ مقاله همزمان در کنفرانس ECCV 2024 تحت نظارت دکتر رهبان

1. Snuffy: Efficient Universal Approximating Whole Slide Image Classification Framework
تبریک به آقای حسین جعفری‌نیا دانشجوی کارشناسی‌ارشد و خانم نهال میرزایی دانشجوی دکترای آزمایشگاه RIML و علیرضا عالی‌پناه، دانیال حمدی و سعید رضوی دانشجویان کارشناسی آزمایشگاه

2. Deciphering the Role of Representation Disentanglement: Investigating Compositional Generalization in CLIP Models
تبریک به آقای رضا عباسی دانشجوی کارشناسی‌ارشد آزمایشگاه RIML
تحت نظارت دکتر رهبان و دکتر سلیمانی


https://www.linkedin.com/posts/mohammad-hossein-rohban-75567677_eccv-activity-7216325744702992386-cQVq?utm_source=share&utm_medium=member_android

RL Research Group at RIML: Join Us!

We are excited to announce the formation of a new research group within RIML, dedicated to advancing the field of Reinforcement Learning. If you're passionate about AI and eager to explore the forefront of RL research, this is the perfect opportunity for you.

To foster collaboration and knowledge sharing, we are launching a weekly Journal Club. Every Tuesday from 3:30 to 5:00 PM, we will gather to discuss the latest research papers and breakthroughs in RL. This is a fantastic chance to deepen your understanding, engage in stimulating discussions, and contribute to the growing body of knowledge in this dynamic field.

This Week's Presentation Paper:

Title: On Representation Complexity of Model-based and Model-free Reinforcement Learning
Link: https://arxiv.org/abs/2310.01706
Presenter: Alireza Nobakht

Join us as we delve into the complexities of model-based versus model-free RL approaches.

Session Details:

Time: Tuesdays, 3:30 - 5:00 PM
Location: Online at https://vc.sharif.edu/ch/rohban
For in-person attendance, please message me on Telegram at @infinity2357.

Stay updated and learn more about our activities by visiting our blog: rljclub.github.io.

We look forward to seeing you and embarking on this exciting research journey together!

#RLJClub #JClub #RIML #SUT #AI #RL

RL Journal Club: This Week's Session

We are pleased to invite you to this week's RL Journal Club session, where we will dive into another fascinating paper in the field of Reinforcement Learning.

Paper for Discussion:

Title: Three Dogmas of Reinforcement Learning 
Link: https://arxiv.org/abs/2407.10583

Join us as we explore and critically analyze the insights presented in this paper. This session promises to be a thought-provoking discussion, providing an opportunity to deepen your understanding of the fundamental concepts and challenges in RL.

Session Details:

Date: Tuesday
Time: 3:30 - 5:00 PM
Location: Online at https://vc.sharif.edu/ch/rohban
For in-person attendance, please message me on Telegram at @infinity2357.

Note: The discussion is open to everyone, but we can only host students of Sharif University of Technology in person.

We look forward to your participation!

#RLJClub #JClub #RIML #SUT #AI #RL

RL Journal Club: This Week's Session

We are excited to invite you to this week's RL Journal Club session, where we will explore an influential paper in the field of Reinforcement Learning. The session will be presented by our professor, Mohammad Hossein Rohban.

This Week's Presentation Paper:

Title: MOReL: Model-Based Offline Reinforcement Learning
Link: https://arxiv.org/abs/2005.05951
Presenter: Professor Mohammad Hossein Rohban

In this session, we will discuss the MOReL framework, which introduces a model-based approach to offline reinforcement learning, aiming to improve the data efficiency and experimental velocity of RL. The paper explores how a pessimistic MDP can be used to safely and effectively train policies using only historical data, offering a fresh perspective on offline RL.

Session Details:

Date: Tuesday
Time: 3:30 - 5:00 PM
Location: Online at https://vc.sharif.edu/ch/rohban
For in-person attendance, please message me on Telegram at @infinity2357.

Note: The discussion is open to everyone, but we can only host students of Sharif University of Technology in person.

We look forward to your participation!

#RLJClub #JClub #RIML #SUT #AI #RL

🧠 RL Journal Club: This Week's Session

🤝 We invite you to join us for this week's RL Journal Club session, where we will dive into the fascinating world of Modular Reinforcement Learning. This session will explore the concept of modular RL, an approach that decomposes complex RL tasks into specialized components to enhance scalability and adaptability.

✅ This Week's Presentation:

🔹 Title: Modular Reinforcement Learning
🔸 Presenter: Arash Marioriyad
🌀 Abstract: Modular RL is an approach that emphasizes the decomposition of complex RL-based learning tasks into modular components. This methodology addresses the scalability and adaptability challenges inherent in traditional reinforcement learning by structuring agents as collections of interacting modules, each specialized for specific sub-tasks or aspects of the environment.

The presentation will be based on the following papers:

▪️ Modular Lifelong Reinforcement Learning via Neural Composition (https://arxiv.org/abs/2207.00429)
▪️ Compete and Compose: Learning Independent Mechanisms for Modular World Models (https://arxiv.org/abs/2404.15109)
▪️ Multi-Task Reinforcement Learning with Soft Modularization (https://arxiv.org/abs/2003.13661)
▪️ Modular Multitask Reinforcement Learning with Policy Sketches (https://arxiv.org/abs/1611.01796)
▪️ Recurrent Independent Mechanisms (https://arxiv.org/abs/1909.10893)

Session Details:

📅 Date: Tuesday
🕒 Time: 3:00 - 4:30 PM
🌐 Location: Online at https://vc.sharif.edu/ch/rohban
📍 For in-person attendance, please message me on Telegram at @infinity2357.

☝️ Note: The discussion is open to everyone, but we can only host students of Sharif University of Technology in person.

💯 This session promises to be an engaging exploration of how modular approaches can transform the scalability and efficiency of reinforcement learning systems. Don't miss this opportunity to deepen your understanding and participate in thought-provoking discussions!

✌️ We look forward to your participation!

#RLJClub #JClub #RIML #SUT #AI #RL

🧠 RL Journal Club: This Week's Session

🤝 We invite you to join us for this week's RL Journal Club session, where we will explore the intriguing synergies between Reinforcement Learning (RL) and Large Language Models (LLMs). This session will delve into how these two powerful fields intersect, offering new perspectives and opportunities for advancement in AI research.

✅ This Week's Presentation:

🔹 Title: Synergies Between RL and LLMs
🔸 Presenter: Moein Salimi
🌀 Abstract: In this presentation, we will review research studies that combine Reinforcement Learning (RL) and Large Language Models (LLMs), two domains that have been significantly propelled by deep neural networks. The discussion will center around a novel taxonomy proposed in the paper, categorizing the interaction between RL and LLMs into three main classes: RL4LLM, where RL enhances LLM performance in NLP tasks; LLM4RL, where LLMs assist in training RL models for non-NLP tasks; and RL+LLM, where both models work together within a shared planning framework. The presentation will explore the motivations behind these synergies, their successes, potential challenges, and avenues for future research.

The presentation will be based on the following paper:

▪️ The RL/LLM Taxonomy Tree: Reviewing Synergies Between Reinforcement Learning and Large Language Models (https://arxiv.org/abs/2402.01874)

Session Details:

📅 Date: Tuesday
🕒 Time: 3:30 - 5:00 PM
🌐 Location: Online at https://vc.sharif.edu/ch/rohban
📍 For in-person attendance, please message me on Telegram at @infinity2357

☝️ Note: The discussion is open to everyone, but we can only host students of Sharif University of Technology in person.

💯 This session promises to be an enlightening exploration of how RL and LLMs can work together to push the boundaries of AI research. Don’t miss this opportunity to deepen your understanding and engage in thought-provoking discussions!

✌️ We look forward to your participation!

#RLJClub #JClub #RIML #SUT #AI #RL #LLM

به دلیل قطعی برق دانشگاه، این جلسه به هفته آینده موکول شد.