Herke van Hoof is currently a postdoctoral fellow at McGill University in Montreal, Canada. At McGill, Herke works with Professors Joelle Pineau, Dave Meger, and Gregory Dudek. Herke works on machine learning for autonomous robots in perceptually challenging environments. For robots to master a wide array of tasks, machine learning is indispensable, but it is equally important that such tasks can be learned in non-standardized and unstructured environments such as homes or hospitals. Learning tasks in such complicated environment puts additional demands on algorithms for machine learning, perception, and control.


One example of such a task is exploring the objects present in a novel environment. Segmenting objects using passive sensing is inherently limited. By interacting with the environment, the robot can improve its understanding of the different objects that are present. However, interaction is costly. By expressing the uncertainty in the robot’s understanding of the world, it becomes possible to select actions based on the information they are expected to yield about the environment, and thus speed up the learning progress.

In another project, we consider reinforcement learning with high-dimensional inputs. Current approaches have usually tried to learn features in a separate step. However, such features cannot be informed by what is relevant for the task at hand. We have taken a complementary approach, where we have developed a non-parametric reinforcement learning method that only depends on the similarity between data-points, independent of the embedding dimensionality.

Currently, I’m working on new ways to exploit known robot models and/or simulators to make reinforcement learning more efficient. I am looking to use a generative model of the robot to characterise its belief over unknown parameters, and pre-training a policy that learns to trade-off exploration and exploitation based on this characterisation.

Before joining McGill, Herke van Hoof obtained his PhD at TU Darmstadt, Germany, under the supervision of Professor Jan Peters, where he graduated in November 2016. Herke got his bachelor and master degrees in Artificial Intelligence at the University of Groningen in the Netherlands.

Recent News

  • ICRA paper accepted

    New paper accepted at ICRA: V. Barbaros, H. van Hoof, A. Abdolmaleki & Dave Meger: A Comparison of Eager and Memory-Based Non-Parametric Stochastic Search Methods for Learning Control. Draft coming soon. Congrats, Victor!

  • JMLR paper accepted

    Our paper “Learning of Non-Parametric Control Policies with High-Dimensional State Features” was accepted for publication in JMLR. A draft is available on the publications page.

  • Machine Learning Journal / ECML paper accepted!

    Our paper “Generalized Exploration in Policy Search” was accepted to be published in the journal track of ECML! A draft is available on the publications page.

An archive of news items can be found on the News page

Key References

Van Hoof, H; Neumann, G; Peters, J

Non-parametric Policy Search with Limited Information Loss

Journal of Machine Learning Research, 18 (73), pp. 1-46, 2017.

Links | BibTeX

van Hoof, Herke ; Tanneberg, Daniel ; Peters, Jan

Generalized Exploration in Policy Search

Machine Learning - Special issue ECML PKDD, 2017, ISSN: 1573-0565.

Links | BibTeX

van Hoof, Herke; Chen, Nutan; Karl, Maximilian; van der Smart, Patrick; Peters, Jan

Stable Reinforcement Learning with Auto-Encoders for Tactile and Visual Data

International Conference on Intelligent Robots and Systems, 2016.

Links | BibTeX

van Hoof, Herke; Kroemer, Oliver; Peters, Jan

Probabilistic Segmentation and Targeted Exploration of Objects in Cluttered Environments

IEEE Transactions on Robotics (TRo), 5 , pp. 1198-1209, 2014.

Links | BibTeX

A full list of publications can be found at the  publications  page.


Projects are coming soon!