State-of-the-art machines to manipulate soft matter —materials that behave as soft solids or liquids, depending on the level of force applied to them, such as sunscreen gels, clay, mayonnaise, tomato sauce— are preprogrammed to execute the same motion endlessly. By itself, the machine has no intelligence, no perception that a soft material must be manipulated, nor can autonomously adapt its plans to change the shape of the material. The intelligence is currently placed in the mind of the developer(s) who built the system and told it how to move. The SMEAR project
aims at bringing part of this intelligence and autonomy in manipulating soft matter into the machine, by integrating and improving the latest available perception technologies in the domain of computer vision and artificial intelligence (AI) with ``traditional'' state-of-the-art model-based engineering techniques, with the goal of designing a prototype robotic system for autonomous manipulation of soft matter
The SMEAR project is part of the multi-disciplinary 'Engineering Design research program'
of the Department of Mechanical Engineering of Eindhoven University of Technology (TU/e). The project is a collaboration between the divisions of Dynamical Systems Design (DSD, focused on systems, dynamics and control), Thermo Fluids Engineering (TFE, focused on heat and flow), and Computational and Experimental Mechanics (CEM, focused on mechanics and materials). The design of a soft-material manipulating robot requires multi-disciplinarity, combining hardware design, modeling, computer vision, AI, programming, and robot control. The most prominent components of the SMEAR project are:
- The design and validation of a perception module able to perceive the spatial occupancy (shape) of the soft material based from RGB(D) camera images, soft matter properties, and enviroment information, potentially including robot-soft-matter interaction forces. Training of the perception module is expected to be based on synthetic data (photorealistic rendering / shape pairs);
- The development of an at-least interactive-rate (60Hz) computational method for soft materials, which captures the essential physics (yield stress, non-Newtonian fluid rheological behavior). A prototype computer code capturing essential continuum-scale physics will be implemented and validated against experiments;
- The implementation of the numerical model in a virtual robot environment, combining a physics engine, photo realistic rendering, and advanced robot control software. Essentially, a digital twin of the robot and material will be constructed, thus creating a virtual 'playground';
- The design, construction and testing of a physical prototype of a robotic system for autonomous manipulation of soft matter, showing perception capability (shape and material property estimation) in the real world.
- Detailed literature review of soft-matter modeling, particle-based methods, deformable and soft object reconstruction from RGB(D) camera images, robot control of soft/granular matter, parameter calibration procedures;
- Development and implementation of the above-mentioned models and methods;
- Pro-actively enabling the collaboration between the involved disciplines;
- Dissemination of the results of your research in international and peer-reviewed journals and conferences;
- Writing a successful dissertation based on the developed research and defending it;
- Assume educational tasks like the supervision of Master students and internships.