115 lines
8.1 KiB
Markdown
115 lines
8.1 KiB
Markdown
tags:: [[#zotero]]
|
||
title:: @icse2025-paper818
|
||
item-type:: [[document]]
|
||
original-title:: icse2025-paper818
|
||
links:: [Local library](zotero://select/library/items/NNFMQPRV), [Web library](https://www.zotero.org/users/1039502/items/NNFMQPRV)
|
||
|
||
- ### Attachments
|
||
- [PDF](zotero://select/library/items/3SZK2A57) {{zotero-imported-file 3SZK2A57, "icse2025-paper818.pdf"}}
|
||
- ### Notes
|
||
- # Annotazioni
|
||
(2/10/2024, 13:55:52)
|
||
|
||
“selecting appropriate models for new local datasets has become a time-consuming and resourceintensive task.” (“icse2025-paper818”, p. 1)
|
||
|
||
“AutoDGR, a novel dual-graph-based model recommendation approach, designed to retrieve the most suitable models by incorporating features across datasets and models.” (“icse2025-paper818”, p. 1)
|
||
|
||
“AutoDGR enhances task and model feature representation by employing semantic task representations and model architectures, effectively capturing relations among datasets and models in separate graphs.” (“icse2025-paper818”, p. 1)
|
||
|
||
“DNNs excel in tackling complex challenges such as visual comprehension, speech recognition, natural language processing, and more.” (“icse2025-paper818”, p. 1)
|
||
|
||
“As a consequence, utilizing pre-trained models (PTMs) has emerged as a more favorable option.” (“icse2025-paper818”, p. 1)
|
||
|
||
“difficult for non-experts to select a model that aligns with their specific requirements.” (“icse2025-paper818”, p. 1)
|
||
|
||
“The issue of model recommendation has thus become urgent for the rapid and widespread deployment of DNN models, aiming to streamline the process for users and conserve resources.” (“icse2025-paper818”, p. 1)
|
||
|
||
“43 types of tasks and hundreds of dataset names available for model selection” (“icse2025-paper818”, p. 1)
|
||
|
||
“the task names are often general and domain-agnostic, lacking the ability to reflect personalized user requirements.” (“icse2025-paper818”, p. 1)
|
||
|
||
“encodes user task descriptions and multimodal dataset samples and then decodes them to embed model features and provide recommendations.” (“icse2025-paper818”, p. 1)
|
||
|
||
“similarities of datasets themselves and ignored the influence of the base architecture of models.” (“icse2025-paper818”, p. 1)
|
||
|
||
“based on the semantics of the user task, we would likely choose the second model as the best fit.” (“icse2025-paper818”, p. 1)
|
||
|
||
“meta-features to enhance model recommendation” (“icse2025-paper818”, p. 2)
|
||
|
||
“First, we propose a semantic graph construction methodology for both datasets and models, utilizing semantic label representation and model architecture information.” (“icse2025-paper818”, p. 2)
|
||
|
||
“This involves establishing two types of relations: semantic relations and structural relations.” (“icse2025-paper818”, p. 2)
|
||
|
||
“Second” (“icse2025-paper818”, p. 2)
|
||
|
||
“we employ separate graph convolutional neural networks with residual connections to embed datasets and models into vector representations according to these two types of relations” (“icse2025-paper818”, p. 2)
|
||
|
||
“Third” (“icse2025-paper818”, p. 2)
|
||
|
||
“297 image classification tasks with datasets from CIFAR-10” (“icse2025-paper818”, p. 2)
|
||
|
||
“two state-of-the-art model recommendation methods.” (“icse2025-paper818”, p. 2)
|
||
|
||
“Moreover, the regression version of AutoDGR offers better generality when model hubs are frequently updated.” (“icse2025-paper818”, p. 2)
|
||
|
||
“task-based and architecturebased meta-features.” (“icse2025-paper818”, p. 2)
|
||
|
||
“predict well-performing models for unseen tasks on frequently changing model hubs.” (“icse2025-paper818”, p. 2)
|
||
|
||
“benchmark dataset” (“icse2025-paper818”, p. 2)
|
||
|
||
“By collecting metadata from various model hubs, the dataset PTMTorrent [10] is proposed to support model reuse [11].” (“icse2025-paper818”, p. 2) These references are very important for our [[PROJECTS/MOSAICO]]
|
||
|
||
“AutoDGR builds a model graph using information from the model hub and does not require extensive computational resource for architecture search, making it a more efficient way to utilize pre-trained models.” (“icse2025-paper818”, p. 2) That's very important.
|
||
|
||
“Model recommendation aims to identify the most suitable pre-trained models for a given task from large model hubs” (“icse2025-paper818”, p. 2)
|
||
|
||
“predicting the performance of hyperparameters on the target dataset” (“icse2025-paper818”, p. 2)
|
||
|
||
“maximize the performance of the model m, denoted as: max m∈M p(m|D). (1)” (“icse2025-paper818”, p. 3)
|
||
|
||
“Directly calculating the performance p(m|D) by running all models on the dataset is time-consuming and resourceintensive.” (“icse2025-paper818”, p. 3)
|
||
|
||
“estimate the performance using the known historical performance of similar models on similar datasets.” (“icse2025-paper818”, p. 3)
|
||
|
||
“Assumption 1: Models with similar parameters, neural architectures, and scales tend to have similar performance on the same dataset.” (“icse2025-paper818”, p. 3) Strong Assumption!!!!!
|
||
|
||
“∝” (“icse2025-paper818”, p. 3) Something wrong with horizontal space
|
||
|
||
“Assumption 2:” (“icse2025-paper818”, p. 3) This is a strong assumption that limits the applicability of the proposed approach. It is important to see some supporting reference for such an assumption.
|
||
|
||
“Models on similar datasets tend to have similar performance.” (“icse2025-paper818”, p. 3)
|
||
|
||
“mi on a new dataset D′ using the historical performance of models on other datasets.” (“icse2025-paper818”, p. 3)
|
||
|
||
“Di and existing datasets Dj,” (“icse2025-paper818”, p. 3)
|
||
|
||
“minimize the difference between the target model mi and the existing models m ̃ on Dj.” (“icse2025-paper818”, p. 3)
|
||
|
||
“By formulating the problem as minimizing the differences between datasets and models, we can apply unified representations to user tasks and models to solve the problem effectively.” (“icse2025-paper818”, p. 3)
|
||
|
||
“By extracting features from datasets and models effectively, it can construct representation graphs for both datasets and models” (“icse2025-paper818”, p. 3)
|
||
|
||
“Then classification and regression methods can be employed on these high-dimensional features to generate the model recommendation results.” (“icse2025-paper818”, p. 3)
|
||
|
||
“To obtain richer semantics from task labels and similar datasets, we construct a graph to share task information.” (“icse2025-paper818”, p. 3)
|
||
|
||
“hsmp(ti) is the representation of samples with label ti” (“icse2025-paper818”, p. 3)
|
||
|
||
“standard sample size” (“icse2025-paper818”, p. 3)
|
||
|
||
“number of samples hcount” (“icse2025-paper818”, p. 3)
|
||
|
||
“of task labels hcat.” (“icse2025-paper818”, p. 4)
|
||
|
||
“hnum(D) = [hsize(D), hcount(D), hcat(D)].” (“icse2025-paper818”, p. 4)
|
||
|
||
“Therefore, we use these output feature vectors as the architectural features during feature extraction.” (“icse2025-paper818”, p. 4)
|
||
|
||
“averaging along the second dimension to obtain a mean feature vector vm” (“icse2025-paper818”, p. 4) Can this represent a bias? Similarly to the possible unbalance of the dataset that you have addressed for the data encoding.
|
||
|
||
“umber of floating-point operations (FLOPs)” (“icse2025-paper818”, p. 5)
|
||
|
||
“ground-truth optimal model” (“icse2025-paper818”, p. 6) How has the ground-truth created? How to ensure that the ground-truth refers the optima model of a given task? The description of the process for the ground-truth creation is necessary.
|
||
|
||
“Considering the scale of data in different scopes, we choose image classification as the evaluation scope for its mature community and large scale of datasets and models” (“icse2025-paper818”, p. 6) This is an important threat to external validity of the approach. It is not clear if the approach works for tasks that are different from that of image classification. |