Auto saved by Logseq

pages/hls__MDE_Intelligence_2023_paper_6_1691144104762_0.md

@@ -0,0 +1,314 @@
+file:: [MDE_Intelligence_2023_paper_6_1691144104762_0.pdf](../assets/MDE_Intelligence_2023_paper_6_1691144104762_0.pdf)
+file-path:: ../assets/MDE_Intelligence_2023_paper_6_1691144104762_0.pdf
+
+- Dagger, NDI, and Curriculum Learning.
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: yellow
+  id:: 64ccd15a-408e-4f38-8272-d7bff85e727a
+- Lastly, we describe what could be done in the future to improve the domain-specific language.
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: green
+  id:: 64ccd763-6629-4359-8f33-0dce185e8695
+- the disadvantage is that model predictive controllers have a high computational cos
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: green
+  id:: 64ccd777-6fc5-4ad9-9253-e1bb59c3ceb2
+- achine learning MPC
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: green
+  id:: 64d15f4d-f217-4c17-a3f8-b0571fc762a9
+- The training of a controller is especially difficult as the controller influences the next states it will visit during its operation. 
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: green
+  id:: 64d15f91-b760-4452-9d4e-b78ad4a0bfe2
+- Complex Training Workflows
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: green
+  id:: 64d15fa0-3041-4a50-b4a6-273536427ac4
+- odel Predictive Contollers
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: green
+  id:: 64d15fa9-279a-4662-baa8-a8370cb1943e
+- the method evaluates the controller and adapts the training data to train a new and hopefully
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: green
+  id:: 64d15ff3-b8f9-4e62-b603-cdf60a44fd4a
+- better controller.
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: green
+  id:: 64d15ff8-7b8f-4899-b475-6f5374fa25a8
+- One of the challenges is data manipulation
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: purple
+  id:: 64d1600f-999c-4701-bd47-18c3000f6ef4
+- control engineers to focus on complex training workflows. 
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: purple
+  id:: 64d16020-59e5-421b-8840-cc04dab1ad8c
+- running exampl
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: green
+  id:: 64d1610d-a401-4e60-b168-10744b4e155f
+- we discuss the background and related work
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: green
+  id:: 64d1611d-f3a5-49b6-996b-3a76a70ce4a5
+- controller for an inverted pendulum
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: green
+  id:: 64d161aa-1fde-462e-83d7-27f5547c0557
+- expert policy to gather training data from
+  ls-type:: annotation
+  hl-page:: 1
+  hl-color:: green
+  id:: 64d161bf-3672-46a1-967f-4f8e43868d74
+- F a force on the cart in the horizontal direction
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d161f2-de8b-42a6-a7e2-99e565da76c0
+- F 2 + 100 ∗ P OS2.
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d16202-4ec3-446c-a1b4-56931e8eb324
+- Model predictive control is a control technique that uses a model to predict the future state of the system and adapts the control actions accordingly
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d1621f-bdbc-4ce3-b809-5d918de33dbf
+- asic model predictive controller
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: purple
+  id:: 64d16236-3af2-4458-9325-7852b9f84d06
+- optimizer
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: blue
+  id:: 64d1623d-71ec-425b-bf30-faec36b1795f
+  hl-stamp:: 1691443775859
+- model
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: blue
+  id:: 64d16242-1cec-4edb-9685-460c58f58cf5
+- , it is interesting to imitate model predictive controllers using machine learning to reduce the computational load of model predictive controller
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d1626b-faa9-45bd-93fb-7fd9f6398962
+- policy
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d162b3-b736-4490-94cc-3283bad84dab
+- mapping between the input and output
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d162c0-d3e4-464a-b4dd-5ba62b7ee2f5
+- expert policy describes a policy that is used to label the data
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: purple
+  id:: 64d162cb-e750-423b-b470-d87db6e612da
+- trainable policy refers to a policy that is constructed using machine learnin
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: purple
+  id:: 64d162da-95e0-4ce3-9a9a-79975527688a
+- technique or function to refer to the building blocks of these methods like the machine learning functions that give trainable policies.
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: purple
+  id:: 64d16328-c2e2-41ef-937b-1bd78097193c
+- Dagger 
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d16335-0fbd-43a9-8331-5c53bd8db10c
+- NDI
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d16337-50f5-4c48-aa15-27fb4c874e68
+- curriculum learning approach
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: yellow
+  id:: 64d16348-224c-4432-a77b-f89fc75ff7c2
+- states in these trajectories are then collected into a dataset and each state is assigned an action from the expert policy. 
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d16386-e98f-48c6-9759-3f58d5dede2a
+- This aggregated dataset is used to train a new policy. This loop is done several times to get a more robust policy
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d163bf-822b-4bd8-bfd2-ad584551d6b2
+- First, the starting points of trajectories are sampled and gathered with their corresponding action given by the expert policy
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d163f5-919c-42fb-adac-7e475197ccae
+- NDI
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: blue
+  id:: 64d1644e-0289-4dbc-a6b0-ac41bf52bef7
+- Curriculum Learning:
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: blue
+  id:: 64d16456-86e2-4332-b7b7-3de07790cfc7
+- the dataset is split into a number of datasets that go from easy to hard.
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d1648f-a163-4c6a-8b6a-49ff6efbc881
+- distance to position zero as a measure of hard a datapoint is to control.
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d164c3-1134-4fc0-a653-ec2ce288b9e2
+- etrain a model when an extension to a dataset occurs
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d164e3-5e8e-47ad-ac24-7e086e23725d
+- training of the policy, selection, and exploration of model
+  ls-type:: annotation
+  hl-page:: 2
+  hl-color:: green
+  id:: 64d16521-5ee2-4ba4-a2b7-1107f8488faa
+- Pandas is a very powerful tool but it is also built very generally. 
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d166c0-ec24-45ca-9a96-2869c24e62de
+- DSL data structures also give more structure. 
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d16718-13d5-4f10-b325-088c650f1b8a
+- [7] Shubhra Kanti Karmaker (“Santu”) et al. “AutoML to Date and Beyond: Challenges and Opportunities”. In: ACM Comput. Surv. 54.8 (Oct. 2021
+  ls-type:: annotation
+  hl-page:: 7
+  hl-color:: green
+  id:: 64d1672d-2f78-4f5a-9838-26d00d3376ba
+- Arbiter is built to train machine learning policies from a singular and static dataset in an ethical way. 
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d16769-1627-473f-9a73-295b4d7e0091
+- create the complex workflows that come with methods like Dagger, NDI, and curriculum learning.
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d167bb-378a-4787-bde0-3f2a8ff85baf
+- mplements the function
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d1683d-5ffe-4b46-8c2e-82fcf1c9cb3a
+- policies how they want.
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d16842-f532-43ce-9e96-4acadccd96fc
+- datapoint, dataset, trace dataset.
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d168b5-29d3-448c-81d0-8718455c2ee1
+- It is initialized using dictionaries or pandas dataframes. It
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d168cb-1935-4694-a937-eb7f69859059
+- ata structure classe
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d2621f-9bf5-4031-a835-cfe0499da11f
+- policy class
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d26223-1495-4391-8de1-aa418de292ef
+- function classe
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d26227-7b0e-409f-bbe3-a8dbf4082c7a
+- tructure of a progra
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d26243-58e5-4a2b-9986-04a7cc94922d
+- ata structure classe
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d26252-295b-474b-8c49-63d2a60ed8e2
+- policy class 
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d2625a-3555-4ece-94eb-0871e11a4139
+- Function classe
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d26267-5bf9-40e1-a79b-3b4026958535
+- The DSL functions and policies always transfer data using the DSL data structures.
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d26317-277b-48fd-96f0-e01cfa70b2b2
+- data management
+  ls-type:: annotation
+  hl-page:: 3
+  hl-color:: green
+  id:: 64d2636f-b0e5-42b0-85e0-80b2abde2c07
+- To calculate the output the arithmetic function swaps the policies it has as input with the output of these policies. DSL policies can be added, subtracted, multiplied, and divided with other DSL policies and numbers. This feature is used in methods like Dagger where a linear combination of the expert policy and the trained policy is asked for.
+  ls-type:: annotation
+  hl-page:: 4
+  hl-color:: yellow
+  id:: 64d26457-f8cc-4b63-abd5-765a06e2d764
+- Train Policy
+  ls-type:: annotation
+  hl-page:: 4
+  hl-color:: green
+  id:: 64d265a7-6bfc-45c9-b3a4-aa62c88a567b
+- Simulate system
+  ls-type:: annotation
+  hl-page:: 4
+  hl-color:: green
+  id:: 64d265ab-55b3-4b09-bbc9-3371dcc098fa
+- Simulate system traces:
+  ls-type:: annotation
+  hl-page:: 4
+  hl-color:: green
+  id:: 64d265d6-0c14-44cb-a6b0-fbf5223ae50f
+- Validate datase
+  ls-type:: annotation
+  hl-page:: 4
+  hl-color:: green
+  id:: 64d265db-0a15-4bf5-ad10-0958ee7dd3cd