27 lines
4.1 KiB
Markdown
27 lines
4.1 KiB
Markdown
collapsed:: true
|
|
type:: [[REVIEWS]]
|
|
tags::
|
|
year:: 2025
|
|
venue:: [[RecSys]]
|
|
full-title:: Investigating Carbon Footprint of Recommender Systems Beyond Training Time: A Reproducibility Study
|
|
date-start:: [[09-06-2025]] - 16:29
|
|
date-submitted::
|
|
external-links::
|
|
status:: [[DONE]]
|
|
deadline-submission::
|
|
file:: [[@RecSys_2025_Reproducibility_711]]
|
|
parent::
|
|
todoist:: https://app.todoist.com/app/task/711-investigating-carbon-footprint-of-recommender-systems-beyond-training-time-a-6c7x6q6cJ5hjvX26
|
|
|
|
- ### [[Comments]]
|
|
- Overall Presentation: Overall, the paper is well written and structured. Both the goal and the proposed solution are clearly described. I have only a few concerns related to missing justifications or explanations. In particular, I am not convinced by the rationale for using a mobile setup, nor why such a configuration was chosen to compare the original environment with the one presented in this work.
|
|
- Technical Soundness: The paper defines interesting research questions, especially the one concerning the inference phase. The methodological pipeline is clearly described in the online material. However, the insights provided for the first research question, while valid, are somewhat obvious and expected (i.e., newer hardware being more efficient). It is necessary to emphasize the takeaway message, why it is important and what can be built on top of it.
|
|
- Reproducibility of methods: The online appendix contains all the artifacts required to reproduce the experiments. I briefly tested the Docker image provided by the authors. Although I could not complete the entire process, I was able to easily start the pipelines by following the authors' instructions.
|
|
- Impact: Carbon footprint in recommender systems is indeed a relevant and timely topic. While some results are insightful (e.g., training/inference trade-offs), others are rather expected and may not be impactful for the community. Nevertheless, the emphasis on validation overhead introduces valuable considerations for future research directions on sustainable recommender systems.
|
|
- Artifacts (Availability and Documentation): An online repository is available at [https://anonymous.4open.science/r/RecSys-2025-Reproducibility-Investigating-Carbon-Footprint-of-Recommender-Systems](https://anonymous[[.]]4open[[.]]science/r/[[RecSys]]-[[2025]]-Reproducibility-Investigating-Carbon-Footprint-of-Recommender-Systems), which includes both the artifacts and the instructions to replicate the experiments. A docker image is also available at [https://hub.docker.com/r/beyondtrainingtime/recsys2025-beyond-training-time-gpu/](https://hub.[[docker]].com/r/beyondtrainingtime/recsys2025-beyond-training-[[time]]-gpu/).
|
|
- Artifacts (Executability): The pipeline is clearly described, and the availability of a Docker image is a valuable addition. However, a Docker Compose file would be expected to simplify the creation and execution of the corresponding container.
|
|
- Review: This paper investigates the sustainability of recommender systems by analyzing not only training but also inference energy costs. The study shows that higher training emissions can, in some cases, lead to lower inference emissions, and it quantifies the cost of validation steps, suggesting the need for further research on experimental practices. I have a few concerns about the work, particularly regarding the following issues and choices:
|
|
- The use of mobile hardware and a Windows setup raises concerns about the fairness of comparisons and the generalizability of the results.
|
|
- Many observations (e.g., differences in energy consumption across hardware setups, high variance in validation overheads) are not surprising and risk being perceived as trivial without deeper insights or stronger empirical grounding.
|
|
- More critical reflection is needed on the implications of these findings: Are they truly actionable? Could a better experimental design (e.g., controlling more environmental variables) have led to clearer conclusions?
|
|
- Despite these limitations, the core idea of the paper, i.e., extending recommender system sustainability beyond training, is interesting and potentially useful to practitioners. |