logseq/pages/hls__MDE_Intelligence_2023_paper_6_1691144104762_0.md

file:: MDE_Intelligence_2023_paper_6_1691144104762_0.pdf file-path:: ../assets/MDE_Intelligence_2023_paper_6_1691144104762_0.pdf

• Dagger, NDI, and Curriculum Learning. ls-type:: annotation hl-page:: 1 hl-color:: yellow id:: 64ccd15a-408e-4f38-8272-d7bff85e727a
• Lastly, we describe what could be done in the future to improve the domain-specific language. ls-type:: annotation hl-page:: 1 hl-color:: green id:: 64ccd763-6629-4359-8f33-0dce185e8695
• the disadvantage is that model predictive controllers have a high computational cos ls-type:: annotation hl-page:: 1 hl-color:: green id:: 64ccd777-6fc5-4ad9-9253-e1bb59c3ceb2
• achine learning MPC ls-type:: annotation hl-page:: 1 hl-color:: green id:: 64d15f4d-f217-4c17-a3f8-b0571fc762a9
• The training of a controller is especially difficult as the controller influences the next states it will visit during its operation. ls-type:: annotation hl-page:: 1 hl-color:: green id:: 64d15f91-b760-4452-9d4e-b78ad4a0bfe2
• Complex Training Workflows ls-type:: annotation hl-page:: 1 hl-color:: green id:: 64d15fa0-3041-4a50-b4a6-273536427ac4
• odel Predictive Contollers ls-type:: annotation hl-page:: 1 hl-color:: green id:: 64d15fa9-279a-4662-baa8-a8370cb1943e
• the method evaluates the controller and adapts the training data to train a new and hopefully ls-type:: annotation hl-page:: 1 hl-color:: green id:: 64d15ff3-b8f9-4e62-b603-cdf60a44fd4a
• better controller. ls-type:: annotation hl-page:: 1 hl-color:: green id:: 64d15ff8-7b8f-4899-b475-6f5374fa25a8
• One of the challenges is data manipulation ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 64d1600f-999c-4701-bd47-18c3000f6ef4
• control engineers to focus on complex training workflows. ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 64d16020-59e5-421b-8840-cc04dab1ad8c
• running exampl ls-type:: annotation hl-page:: 1 hl-color:: green id:: 64d1610d-a401-4e60-b168-10744b4e155f
• we discuss the background and related work ls-type:: annotation hl-page:: 1 hl-color:: green id:: 64d1611d-f3a5-49b6-996b-3a76a70ce4a5
• controller for an inverted pendulum ls-type:: annotation hl-page:: 1 hl-color:: green id:: 64d161aa-1fde-462e-83d7-27f5547c0557
• expert policy to gather training data from ls-type:: annotation hl-page:: 1 hl-color:: green id:: 64d161bf-3672-46a1-967f-4f8e43868d74
• F a force on the cart in the horizontal direction ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d161f2-de8b-42a6-a7e2-99e565da76c0
• F 2 + 100 ∗ P OS2. ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d16202-4ec3-446c-a1b4-56931e8eb324
• Model predictive control is a control technique that uses a model to predict the future state of the system and adapts the control actions accordingly ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d1621f-bdbc-4ce3-b809-5d918de33dbf
• asic model predictive controller ls-type:: annotation hl-page:: 2 hl-color:: purple id:: 64d16236-3af2-4458-9325-7852b9f84d06
• optimizer ls-type:: annotation hl-page:: 2 hl-color:: blue id:: 64d1623d-71ec-425b-bf30-faec36b1795f hl-stamp:: 1691443775859
• model ls-type:: annotation hl-page:: 2 hl-color:: blue id:: 64d16242-1cec-4edb-9685-460c58f58cf5
• , it is interesting to imitate model predictive controllers using machine learning to reduce the computational load of model predictive controller ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d1626b-faa9-45bd-93fb-7fd9f6398962
• policy ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d162b3-b736-4490-94cc-3283bad84dab
• mapping between the input and output ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d162c0-d3e4-464a-b4dd-5ba62b7ee2f5
• expert policy describes a policy that is used to label the data ls-type:: annotation hl-page:: 2 hl-color:: purple id:: 64d162cb-e750-423b-b470-d87db6e612da
• trainable policy refers to a policy that is constructed using machine learnin ls-type:: annotation hl-page:: 2 hl-color:: purple id:: 64d162da-95e0-4ce3-9a9a-79975527688a
• technique or function to refer to the building blocks of these methods like the machine learning functions that give trainable policies. ls-type:: annotation hl-page:: 2 hl-color:: purple id:: 64d16328-c2e2-41ef-937b-1bd78097193c
• Dagger ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d16335-0fbd-43a9-8331-5c53bd8db10c
• NDI ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d16337-50f5-4c48-aa15-27fb4c874e68
• curriculum learning approach ls-type:: annotation hl-page:: 2 hl-color:: yellow id:: 64d16348-224c-4432-a77b-f89fc75ff7c2
• states in these trajectories are then collected into a dataset and each state is assigned an action from the expert policy. ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d16386-e98f-48c6-9759-3f58d5dede2a
• This aggregated dataset is used to train a new policy. This loop is done several times to get a more robust policy ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d163bf-822b-4bd8-bfd2-ad584551d6b2
• First, the starting points of trajectories are sampled and gathered with their corresponding action given by the expert policy ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d163f5-919c-42fb-adac-7e475197ccae
• NDI ls-type:: annotation hl-page:: 2 hl-color:: blue id:: 64d1644e-0289-4dbc-a6b0-ac41bf52bef7
• Curriculum Learning: ls-type:: annotation hl-page:: 2 hl-color:: blue id:: 64d16456-86e2-4332-b7b7-3de07790cfc7
• the dataset is split into a number of datasets that go from easy to hard. ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d1648f-a163-4c6a-8b6a-49ff6efbc881
• distance to position zero as a measure of hard a datapoint is to control. ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d164c3-1134-4fc0-a653-ec2ce288b9e2
• etrain a model when an extension to a dataset occurs ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d164e3-5e8e-47ad-ac24-7e086e23725d
• training of the policy, selection, and exploration of model ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64d16521-5ee2-4ba4-a2b7-1107f8488faa
• Pandas is a very powerful tool but it is also built very generally. ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d166c0-ec24-45ca-9a96-2869c24e62de
• DSL data structures also give more structure. ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d16718-13d5-4f10-b325-088c650f1b8a
• [7] Shubhra Kanti Karmaker (“Santu”) et al. “AutoML to Date and Beyond: Challenges and Opportunities”. In: ACM Comput. Surv. 54.8 (Oct. 2021 ls-type:: annotation hl-page:: 7 hl-color:: green id:: 64d1672d-2f78-4f5a-9838-26d00d3376ba
• Arbiter is built to train machine learning policies from a singular and static dataset in an ethical way. ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d16769-1627-473f-9a73-295b4d7e0091
• create the complex workflows that come with methods like Dagger, NDI, and curriculum learning. ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d167bb-378a-4787-bde0-3f2a8ff85baf
• mplements the function ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d1683d-5ffe-4b46-8c2e-82fcf1c9cb3a
• policies how they want. ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d16842-f532-43ce-9e96-4acadccd96fc
• datapoint, dataset, trace dataset. ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d168b5-29d3-448c-81d0-8718455c2ee1
• It is initialized using dictionaries or pandas dataframes. It ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d168cb-1935-4694-a937-eb7f69859059
• ata structure classe ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d2621f-9bf5-4031-a835-cfe0499da11f
• policy class ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d26223-1495-4391-8de1-aa418de292ef
• function classe ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d26227-7b0e-409f-bbe3-a8dbf4082c7a
• tructure of a progra ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d26243-58e5-4a2b-9986-04a7cc94922d
• ata structure classe ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d26252-295b-474b-8c49-63d2a60ed8e2
• policy class ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d2625a-3555-4ece-94eb-0871e11a4139
• Function classe ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d26267-5bf9-40e1-a79b-3b4026958535
• The DSL functions and policies always transfer data using the DSL data structures. ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d26317-277b-48fd-96f0-e01cfa70b2b2
• data management ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64d2636f-b0e5-42b0-85e0-80b2abde2c07
• To calculate the output the arithmetic function swaps the policies it has as input with the output of these policies. DSL policies can be added, subtracted, multiplied, and divided with other DSL policies and numbers. This feature is used in methods like Dagger where a linear combination of the expert policy and the trained policy is asked for. ls-type:: annotation hl-page:: 4 hl-color:: yellow id:: 64d26457-f8cc-4b63-abd5-765a06e2d764
• Train Policy ls-type:: annotation hl-page:: 4 hl-color:: green id:: 64d265a7-6bfc-45c9-b3a4-aa62c88a567b
• Simulate system ls-type:: annotation hl-page:: 4 hl-color:: green id:: 64d265ab-55b3-4b09-bbc9-3371dcc098fa
• Simulate system traces: ls-type:: annotation hl-page:: 4 hl-color:: green id:: 64d265d6-0c14-44cb-a6b0-fbf5223ae50f
• Validate datase ls-type:: annotation hl-page:: 4 hl-color:: green id:: 64d265db-0a15-4bf5-ad10-0958ee7dd3cd