{"id":1967,"date":"2018-03-25T12:07:32","date_gmt":"2018-03-25T10:07:32","guid":{"rendered":"https:\/\/blog.idethloff.de\/wordpress\/?p=1967"},"modified":"2018-05-06T13:44:22","modified_gmt":"2018-05-06T11:44:22","slug":"making-of-ids-vr-app","status":"publish","type":"post","link":"https:\/\/blog.idethloff.de\/wordpress\/?p=1967","title":{"rendered":"Making of &#8230; IDs VR App"},"content":{"rendered":"<p><strong>Making of &#8211; A Virtual Reality App on Android for Google Cardboard with Unity 3D<\/strong><\/p>\n<p><strong>Motivation<\/strong><br \/>\nI\u2019ve been interested in Mobile VR (that means low-cost VR without an expensive VR headset and without a high-end gaming PC) for some time. Last year, I took the UC San Diego edX course \u201cHow Virtual Reality works\u201d and learned a lot in theory and by testing and reflecting about Google Cardboard apps during the weekly assignments. I thought, it would be absolutely fascinating to create something that really works on my Android smartphone and relates to the aspects we got into in theory last year, so I also took this years edX course \u201cCreating Virtual Reality (VR) Apps\u201d (CSE190x). I wanted to see how difficult it is and if it can be done without massive programming skills.<\/p>\n<p><strong>The result<\/strong><br \/>\n<img loading=\"lazy\" decoding=\"async\" class=\"alignleft\" src=\"https:\/\/www.idethloff.de\/IDsVRapp\/VR-logo.jpg\" alt=\"Logo IDs VR App\" width=\"102\" height=\"152\" \/>Thanks to the edX UC San Diego course \u201cCreating Virtual Reality (VR) Apps\u201d there is an app, which actually works \u2026 This is Build 1.16 (March 22nd, 2018). With Google Cardboard, you don\u2019t have (hand) controllers and therefore interaction happens by gaze and one button. Nevertheless, I\u2019m quite impressed what can be done.<br \/>\nIf you actually want to create an app, I warmly recommend this 6-week-course which is part three of a \u201c<a href=\"https:\/\/www.edx.org\/professional-certificate\/virtual-reality-vr-app-development\" target=\"_blank\" rel=\"noopener\">Professional Certificate Program<\/a>\u201d.<br \/>\n<iframe loading=\"lazy\" src=\"https:\/\/www.youtube-nocookie.com\/embed\/kMQ2O0dTejU\" width=\"560\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<p>If you would like to try IDs VR App on your Android Smartphone at your own risk, here is the download link: <a href=\"https:\/\/www.idethloff.de\/IDsVRapp\/Build16_test.apk\" target=\"_blank\" rel=\"noopener\">https:\/\/www.idethloff.de\/IDsVRapp\/Build16_test.apk<\/a><\/p>\n<p><strong>How it began<\/strong><br \/>\nYou need a current Android smartphone, a Google Cardboard and a PC on which runs Unity 3D (<a href=\"https:\/\/unity3d.com\/de\" target=\"_blank\" rel=\"noopener\">https:\/\/unity3d.com\/de<\/a>) \u2013 there is a personal free edition \u201cFor beginners, students and hobbyists who want to explore and get started with Unity.\u201d Btw, Unity 3D is one of three well-known VR authoring tools, the other ones are Lumberyard and Unreal Engine.<br \/>\nIn order to test at any one time what you\u2019ve created in Unity 3D, you need to connect the smartphone with the PC and allow some configuration options as \u201cdeveloper\u201d.<br \/>\nIt was a little bit tricky to install and configure the necessary Unity 3D environment and its many components: GoogleVRForUnity, Android SDK and JDK. Installing Android SDK (now via &#8222;Android Studio&#8220; and afterwards the SDK component) was awful because it took a long long time and when I tried &#8222;Build and Run&#8220; in Unity, I just got the error message &#8222;CommandInvokationFailure: Unable to list target platforms&#8220; &#8211; Using the older version of JDK didn&#8217;t help either &#8211; at least until I deinstalled the Java 9 SDK. &#8222;Build and Run&#8220; then led to numerous other error messages during which I had to change the player settings in Unity (Minimal API level, Package name) and had to change settings on my smartphone which I had to connect via USB during the process (enabling developer options). It was a very nice moment, when I saw the Unity icon in the Cardboard app on my smartphone and could see what I had created in Unity (which was just a room with four walls and a red point light)!<\/p>\n<p>This animated gif shows the difference between week 1 (kind of &#8222;Hello World&#8220;-moment) and week 6 (finished Test-App):<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.idethloff.de\/IDsVRapp\/VR-start-ani.gif\" alt=\"Animated Gif showing the difference between week 1 and week 6\" \/><\/p>\n<p><strong>Week 1 &#8211; Game Objects, Camera and Lighting<\/strong><br \/>\nGame Objects \/ Scenery: You want to have some content in your app and there are a few possibilities for obtaining\/placing 3D models: You can create them directly in Unity 3D or you can fetch them from the Unity Asset Store (there are many free ones) or you can import them, after you\u2019ve created them in another application (e.g. 3D modeling tool). You have to think about reducing the complexity of the objects (polygons in the objects and pixels in the textures) so that the smartphone can keep up with its rendering power (which is much lower than that of a gaming PC). My app became bigger in size each week and the first thing, which blew up its size was a real-world-object which I scanned with the iPad TRNIO app (<a href=\"http:\/\/trn.io\/2\/2G70hcJJJU\/\" target=\"_blank\" rel=\"noopener\">result shown on TRNIO web page<\/a>) and afterwards converted in order to get a format which worked in Unity. Luckily, there is a statistics option in Unity, where you can see in Play Mode if you still get the recommended rate of at least 60fps.<br \/>\nThis animated gif shows a picture of the original and of the 3D scan result of my hippo (in the app you can walk around it):<br \/>\n<img decoding=\"async\" src=\"https:\/\/www.idethloff.de\/IDsVRapp\/VR-nili-ani.gif\" alt=\"Animated Gif showing photo of hipp in real world and hippo as 3D scan\" \/><\/p>\n<p>Lighting: You\u2019ve got 4 light types in Unity: Ambient Light, Directional Light, Point Light, Spot Light \u2013 all these are things which you can do in Unity directly via the menus and it\u2019s quite easy to add light sources like spotlight or point light and change color, range, intensity.<br \/>\nCamera: Depending on what\u2019s in your VR world, when you want motion on your screen, you can either move the Unity virtual camera or move the world.<\/p>\n<p><strong>Week 2 \u2013 Gaze interaction<\/strong><br \/>\nActually, it isn\u2019t gaze interaction, but sensors in the smartphone detect the movement of the head and act accordingly. Triggering an action with the \u201cgaze cursor\u201d (which was drawn in the center of the user\u2019s view) can either be done by button on the headset (we did that in our app) or by the \u201cstare and wait\u201d method (which would be installed as a circle which fills out within a few seconds if you hold the gaze).<br \/>\nThat was the moment when our scripting started: During the course we wrote 9 C#-scripts from scratch. I would have been totally lost, if there hadn\u2019t been two enthusiastic course assistants who had pre-recorded a lot of video tutorials and showed step-by-step what had to be done.<\/p>\n<p><strong>Week 3\/4\/5\/6 \u2013 UI and locomotion, selection and manipulation of objects, wayfinding, textures, spatial audio<\/strong><br \/>\nWeek 3\/4\/5\/6 were a lot about menus in 3D, moving in 3D and how to avoid motion sickness (because in VR, \u201cwhat you see is not what you get\u201d). Our menu was installed as a UI with buttons, which were combined with scripts in order to make something happen. This meant: Making objects gazeable\/transformable, instantiating prefab game objects like furniture by looking at the floor and moving around in the 3D world. Locomotion was technically realized as teleporting (e.g. moving from one spot in the 3D world to another spot instantly \u2013 a little bit of magic) and as walking. As we had constructed a room with 4 walls, the walking had to be limited by script but it was a nice touch to allow a view through the walls (That\u2019s why the skybox-feature with a 360 photo made sense). Teleportation avoids motion sickness but can also be a bit disorienting. Therefore, later on, we added a minimap of our room by adding an additional camera with projecting its view as texture on a second UI. It\u2019s quite interesting to see the minimap updating automatically when objects are added or the user (which is represented as a sphere) moves through the room. It was important to make the minimap UI and the other UI user-centered and draggable in order to keep them useful when moving around in the room. For the implementation of these features we needed a lot of new scripts \u2026 In comparison, it was very easy to decorate a little bit by giving new textures to the game objects \u2013 it was nice to use own photos, import them into Unity and just drop them onto the chosen objects in the scene view \u2013 however, what I did wasn\u2019t ideal because the size of the app increased a lot &#8230; It was also very relaxing to add spatial audio (e.g. a radio game object and a sound file combined with some configurations in Unity) in order to get more realism: You need a headset with your smartphone to identify if the sound comes from the left or right, but it\u2019s very impressive. We also covered the topic \u201cpitfalls to avoid\u201d (among them \u201ccreating a VR app that doesn\u2019t actually require VR\u201d, \u201cmaking (wrong) assumptions about users\u201d or technical problems like \u201cvisual lag\u201d) and the necessity of usability testing (test for functionality &amp; test for comfort) as well as user training. One way of user training would be a demo video and I opted for a screen recording while using the cardboard instead of recording during the Game View Play mode in Unity on my PC.<\/p>\n<p>This picture shows my project in the Unity 3D interface:<br \/>\n<a href=\"https:\/\/www.idethloff.de\/IDsVRapp\/VR-wo6-unity.jpg\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/www.idethloff.de\/IDsVRapp\/VR-wo6-unity-kl.jpg\" alt=\"Unity 3D Oberfl\u00e4che\" \/><\/a><br \/>\nA lot of testing happened in Unity itself with the mouse moving around for simulating head movement:<\/p>\n<p><iframe loading=\"lazy\" src=\"https:\/\/www.youtube-nocookie.com\/embed\/N0k30nKraWI\" width=\"560\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<p><strong>My conclusion<\/strong><br \/>\nYes, it was possible to create a VR app which is working (mostly, there are some bugs), but I couldn\u2019t have done it without the before-mentioned video tutorials of CSE190x. It was fun, it was hard work and I appreciate the work of the MOOC team of Prof. Schulze, UC San Diego. The course was very good in discussing and showing step-by-step the realization of important concepts in Mobile VR.<br \/>\n(<a href=\"https:\/\/courses.edx.org\/certificates\/31c82c18ba214050ab5a2f142a92c336\" target=\"_blank\" rel=\"noopener\">My verified CSE190x course certificate<\/a>)<br \/>\nCreating a specific VR app just isn\u2019t possible without programming skills although Unity 3D already does a lot of good work. However, in my opinion, it would be difficult to implement new things just by searching in Unity 3D web pages or forum entries \u2013 at least, this is where I failed, when I looked around a little bit for additional features.<\/p>\n<p><strong>Didactical implications for Mobile VR<\/strong><br \/>\nIt would be \u201ccomparatively easy\u201d to implement a scenery where students could explore their surroundings and experience a (hopefully immersive) 3D world. However, you would have to create convincing 3D models of the chosen topic and implement interactions so it doesn\u2019t get boring. I am still surprised\/impressed what you can do just with gaze interaction: adding prefab objects and transforming them offers a wide range of possibilities and not just a kind of \u201cinterior design app\u201d. Gaze interaction isn\u2019t as intuitive as hand controllers would be, but it can be learned fast (even without the help of a demo video or a tutorial). Good to know: Using text in 3D isn\u2019t a good idea, because you can\u2019t really read it, e.g. you have to do without it and maybe instead use sound for information\/instructional purposes. Unfortunately, I\u2019ve seen many VR apps where you have a considerable lag and see pixels \u2013 there\u2019s no benefit in those apps, where either the loading via Internet or just the smartphone are too slow. It will be interesting to see the progress in technical development \u2013 maybe creating VR apps someday will be as easy as creating a homepage \u2013 who still remembers writing html code? Most teachers are lost, when I tell them about the \u201cswitch to html\u201d-icon in the Moodle text editor.<br \/>\nAnd who could write and understand C# scripts which are necessary for creating VR apps with Unity 3D? Therefore, creating VR apps which exceed showing a 3D world fitted with very simple objects and where you can move around a little bit, at present, will become very expensive in production. And still, you couldn\u2019t compete with the perfect graphics &amp; range of interactions in games on Sony PS4 etc. \u2026 Last but not least: \u201cDon\u2019t create a VR app that doesn\u2019t actually require VR\u201d.<br \/>\nNevertheless, I\u2019d recommend that you look at some existing VR apps and be impressed and enjoy the journey. It\u2019s an intriguing experience, especially if you see it for the first time.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Making of &#8211; A Virtual Reality App on Android for Google Cardboard with Unity 3D Motivation I\u2019ve been interested in Mobile VR (that means low-cost VR without an expensive VR headset and without a high-end gaming PC) for some time. Last year, I took the UC San Diego edX course \u201cHow Virtual Reality works\u201d and &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/blog.idethloff.de\/wordpress\/?p=1967\" class=\"more-link\"><span class=\"screen-reader-text\">\u201eMaking of &#8230; IDs VR App\u201c <\/span>weiterlesen<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2,4,18],"tags":[],"class_list":["post-1967","post","type-post","status-publish","format-standard","hentry","category-e-learning","category-software","category-vr"],"_links":{"self":[{"href":"https:\/\/blog.idethloff.de\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/1967","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.idethloff.de\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.idethloff.de\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.idethloff.de\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.idethloff.de\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1967"}],"version-history":[{"count":22,"href":"https:\/\/blog.idethloff.de\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/1967\/revisions"}],"predecessor-version":[{"id":1990,"href":"https:\/\/blog.idethloff.de\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/1967\/revisions\/1990"}],"wp:attachment":[{"href":"https:\/\/blog.idethloff.de\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1967"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.idethloff.de\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1967"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.idethloff.de\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1967"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}