Schlagwort: ‘Task Planning’
Robot Cooking – Transferring observations into a planning language
Transferring observations into a planning language: An automated approach in the field of cooking
In the Robot Cooking project, an automated method is developed to analyze and identify motion data and convert it into a machine-readable planning language. This is done using a cooking scenario as an example in which the motion data is captured by recording the hand pose of the cook.
The recording is done using a motion capture system consisting of seven cameras and a glove with three markers on the back of the chef’s hand. The position of the markers is determined by triangulation. This provides enough information to derive the hand pose. The recording is done at 120 frames per second. Before the cooking process, all objects in the workspace are identified and their initial positions determined. Motion data is continuously recorded and converted into poses with time stamps. Additional information such as velocity, acceleration and angle in relation to the tabletop are derived from the raw data.
A initial structure of the dataset is created by finding the side actions using classification. Here, pick, move and place are identified as recurrent side actions. A separate training dataset is used to train a classifier that recognizes these actions. This enables an easier analysis of the remaining actions.
Clustering is applied to identify unknown actions. A dynamic approach allows analysis despite high variability in execution. A unique fingerprint for each action is found, based on the orientation of the back of the hand and its speed on the table plane, to assign each frame to a cluster and finally to an action.
The knowledge gained from classification and clustering is translated into a machine-readable Planning Domain Definition Language (PDDL). A schedule is created, with known actions directly assigned. Start and end positions are specified, and virtual object tracking is used to represent the progression of objects during cooking. For unknown actions, preconditions and effects are handled dynamically. The results are translated into a machine-readable PDDL. This formal representation enables efficient automatic scheduling and execution of the previously demonstrated cooking task.
Additional information is available in the video linked above, the poster and the paper.
Contact:
Markus Schmitz
A multi-layered task sequencing approach
Cobots are highly sought by manufacturing companies in contrast to fully automated production lines, as they provide the additional benefit of flexible operations. A major hurdle with current collaborative setups is tedious setup times for efficient and robust co-working as well as poor support for random interruptions.
This project focuses on enabling autonomous collaborative operations for serial manipulators where interruptions from human agents occur at random while ensuring minimal setup times. To this end, two primary aspects that impact task execution are addressed, namely execution time and co-working as enumerated below:
- A method is developed to minimise the total distance travelled, by following the most optimal sequence for the given task while retaining online operational capabilities
- A real-time replanning and co-working algorithm for randomly interruptive environments is developed and implemented to ensure continued operation even when regions of the workspace are occluded while guaranteeing safety of the human agents in the workspace. The co-working controller operates fully autonomously.
An example of the working of the deployed on a prototype platform consisting of a collaborative UR10e arm, a stereo camera for static environment mapping and a laser scanner for mapping of dynamic obstacles is shown in the video.
Contact: Daniel Gossen
music: madiRFAN – Both of Us (https://pixabay.com/music/beats-madirfan-both-of-us-14037/)
Watch the video on our Youtube channel: here.
IGMR-Seminar 11.05.2021, 16:00 – 17:00 Uhr: Task Planning, Environment Representation and Reasoning in Agricultural and Industrial Robotics
Wir freuen uns Oscar Lima vom DFKI (Deutsches Forschungszentrum für Künstliche Intelligenz) aus Osnabrück als Vortragender beim nächsten Termin der IGMR Vortragsreihe im Sommersemester 21 zu haben. Der Titel seines Vortrags lautet Task Planning, Environment Representation and Reasoning in Agricultural and Industrial Robotics.
The focus of the talk will be on DFKI Osnabrück projects. Most of our work is related to agricultural robotics, perception, environment representation, reasoning and task planning. We start the talk with the concept of precision farming, how robots can assist there, to then look into route planning, environment representation, and some of its applications in navigation and expert systems. At the end of the talk we will finish with projects that are related with industry 4.0 and one which aims to provide a generic tool for AI planning in Europe. The talk is light and conceptual, I hope to catch your attention with interesting and new ideas!
Zoom Meeting Informationen:
11.05.2021, 16:00 – 17:00 Uhr
https://rwth.zoom.us/j/95798557131?pwd=WDNoUmxLV3h4R2JTWVZWMWNSajhNZz09
Meeting-ID: 957 9855 7131
Kenncode: 917617
Die Datenschutzhinweise zur Nutzung von Zoom und eine Handreichung für Teilnehmer (Studierende) können von den Seiten des CLS der RWTH Aachen University heruntergeladen werden.
Die Veranstaltungen werden in Zusammenarbeit mit dem VDI-GPP-Arbeitskreis des Bezirksvereins Aachen durchgeführt.
