The Course Project is worth a significant portion of your grade. It offers you the chance to apply your newly acquired skills towards an in-depth project. Your project can be on any topic of your choice related to reinforcement learning. Ideal projects develop new algorithms and/or theoretical results and make good progress towards a new research contribution (e.g. that could be publishable at a conference).
We have put some suggested projects here
To inspire more ideas, consider browsing the RL publications from:
- RLDM: Multi-disciplinary Conference on Reinforcement Learning and Decision Making
- AAMAS: International Conference on Autonomous Agents and MultiAgent Systems
- NIPS: Neural Information Processing Systems
- ICML: International Conference on Machine Learning
- ICLR: International Conference on Representations
- arXiv: e-prints archive
- Stanford AI Group: New and relavent papers from local faculty
- Kaggle challenges: An online machine learning competition website
- 11:59pm Thursday May 4: Initial project proposal.
- 11:59pm Monday May 22: Project milestone due.
- Poster presentations 1:30-4:30pm Wednesday June 7. You only required to participate from 1:30-2:50pm (normal class time) but are encouraged to stay for the whole time.
- Final report due 11:59pm Friday June 9.
Final Project: 32% of total grade
poster: 5% (+2% bonus for best few posters)
The project proposal should include the project team memebrs and the project mentor (someone who agrees to give you feedback). The mentor can be one of the course staff or someone external to the class. It should also include a brief overview of the proposed project and project plan that includes the following (200-400 words):
- What is the problem that you will be investigating? Why is it interesting?
- If relevant, what data or simulator or real RL domain will you be looking at? If you are collecting new datasets, how do you plan to collect them?
- What method or algorithm or theoretical analysis are you proposing? If there are existing implementations, will you use them and how? How do you plan to improve or modify such implementations? If a theoretical question, how plan to make progress?
- What reading will you examine to provide context and background?
- How will you evaluate your results? Qualitatively, what kind of results do you expect (e.g. plots or figures)? Quantitatively, what kind of analysis will you use to evaluate and/or compare your results (e.g. what performance metrics or statistical tests)?
Submission: One member on your team should submit your project proposal using the google form.
Your project milestone report should be between 2 - 3 pages using ICML template
. The following is a suggested structure for your report:
- Title, Author(s)
- Introduction: this section introduces your project, why it’s important or interesting.
- Approach: Describe the current steps you have done. If you are implementing an algorithm, you should have started implementation and ideally have some early stage results. Describe precisely the remaining work you expect to complete.
Submission: Please upload a PDF file named
<your SUNet ID>_milestone.pdf using Gradescope. Note that, each individual in a team is required to make submission (i.e. the same PDF) for grading purpose. The late days are counted by the timestamp of the last submission in the team.
Your final write-up should be between 6 - 8
pages using ICML template
. After the class, we will post all the final reports online so that you can read about each others' work. If you do not want your writeup to be posted online, then please let us know when you submit your writeup.
You should include a brief statement on the contributions of different members of the team. Team members will normally get the same grade, but we reserve the right to differentiate in egregious cases.
Submit your final submission as instructed below:
- A PDF file of your final report submitted through Gradescope.
- A zip file with supplementary materials through our submission script. You are required to include all the code for your project in the supplementary materials.
. The following is a suggested structure for the report:
- Title, Author(s)
- Abstract: It should not be more than 300 words
- Introduction: this section introduces your problem, and the overall plan for approaching your problem
- Background/Related Work: This section discusses relevant literature for your project
- Approach:Algorithm used or developed
- Theoretical results (if relevant): Include assumptions, proof sketches
- Experiment results (if relevant): Details on experiments done. The goal is to describe in enough detail that the results are reproducible.
- Conclusion: What have you learned? Suggest future ideas.
- References: This is absolutely necessary.
is not counted toward your 6-8 page limit.
Examples of things to put in your supplementary material:
- Source code (required).
- Full proof details (if doing a project with theoretical results).
- Cool videos, interactive visualizations, demos, etc. (optional)
Examples of things to not put in your supplementary material:
- All of a submodules (Theano, Caffe, CoreNLP) source code.
- Any code that is larger than 1MB.
- Model checkpoints.
- A computer virus.
You can work in teams of up to 3
people. Larger teams are expected to do larger projects and if you are doing this project to count for this and another class, we expect the project to be larger in scope. See the grading page
for more details.
You may use any existing code, libraries, etc. and consult and any papers, books, online references, etc. for your project. However, you must cite your sources in your writeup and clearly indicate which parts of the project are your contribution and which parts were implemented by others. Under no circumstances may you look at another group’s code or incorporate their code into your project.
If you are doing this project jointly with another class, you must inform us and the other instructors, specify if the are other partners that are not in CS234 that you are working with and also be able to describe the aspects of the project that are relevant to CS234.
The format and guidelines in this page are largely drawn from the project page of CS231n.