An important step in the construction of novel objects is the ability to recognise combinations of raw materials which are likely to be useful. We aim to exploit the intuition that the visual characteristics of candidate raw materials provide useful cues to their potential combinations. Toward this end, we present a Siamese neural network based model that is able to recognise unseen raw materials present in objects given a list of candidate material images. We demonstrate the utility and efficacy of our model within two domains. The first being a proof-of-concept within Minecraft where we predict the combinations of objects that will result in a target object. The second, more realistic domain, uses the ShapeNet 3D model dataset where we attempt to recover the materials present in a model. We empirically demonstrate that our model is able to learn from a subset of object material pairs and generalise to unseen objects, materials, texture packs. Under these conditions of high visual variation, we show that our model outperforms chance and baseline methods.
Reference:
Perlow, J. et al. 2017. Raw material selection for object construction. PRASA-RobMech International Conference, Bloemfontein, South Africa, 29th November - 1 December 2017.
Perlow, J., Rosman, B. S., Hayes, B., & Ranchod, P. (2017). Raw material selection for object construction. http://hdl.handle.net/10204/9889
Perlow, J, Benjamin S Rosman, B Hayes, and P Ranchod. "Raw material selection for object construction." (2017): http://hdl.handle.net/10204/9889
Perlow J, Rosman BS, Hayes B, Ranchod P, Raw material selection for object construction; 2017. http://hdl.handle.net/10204/9889 .
Paper presented at the PRASA-RobMech International Conference, Bloemfontein, South Africa, 29th November - 1 December 2017. This is the accepted version of the paper.