Vuforia / MergeVR Integration

Posted on March 31, 2016 by John Gillespie

This post outlines the steps required to integrate Vuforia for Digital Eyeware with the MergeVR SDK in Unity. The result of this integration will be an Augmented Reality demo app that can be run in the MergeVR headset on your Android device. It will recognize an image marker and display a 3d object on top of that marker, and allow the user to trigger a virtual button on the object – then enter in VR mode and move around the VR scene using the MergeVR headset capactive input buttons.

https://developer.qualcomm.com/software/vuforia-augmented-reality-sdk

http://mergevr.com/

Requirements

Android mobile device

Unity3d – version 4.6.*

Vuforia – Download the core Vuforia unity package – vuforia-unity-5-0-5 (you must be a registered Vuforia developer before the download) and the eyeware samples vuforia-samples-eyewear-unity-5-0-5

MergeVR – download latest Merge SDK Unity package- MergeVR_version_v062

Steps

Create a new empty Unity Project (4.6.*)

Import the Merge SDK Unity package (MergeVR_version_v062.unitypackage)

Import the Vuforia core Unity package (vuforia-unity-5-0-5.unitypackage)

Import the Vuforia Digital Eyeware sample AR/VR package from the vuforia-samples-eyewear-unity-5-0-5.zip file (arvr-5-0-5.unitypackage)

In the Scenes folder find the Scene ‘Vuforia-3-AR-VR’ – Duplicate it (Edit->Duplicate) and rename the new scene ‘DemoVRAR’

Open ‘DemoVRAR’ scene.

Expand the ‘UserHead’ object and then select ‘ARCamera’ – enter your Vuforia license key into the ‘App License Key’ field of the inspector. (https://developer.vuforia.com/license-manager)

Run the scene in Unity Editor mode to confirm everything is working correctly. This sample scene uses the stones target marker (available at https://developer.vuforia.com/sites/default/files/sample-apps/targets/imagetargets_targets.pdf). The app should recognize the stones image marker in your camera and display the mountain and trees 3d object.

Stop running the scene (if you haven’t already)

Find the MergeVR -> Prefabs folder. Drag the MergeCameraController and MergeSDK prefabs into the scene Hierarchy.

Expand the MergeCameraController tree.

Click on ARCamera under UserHead and view the Inspector Pane. Check (enable) the ‘Bind Alternate Camera’ property.

Drag the MergeCameralController root transform to the ARCamera property ‘Central Anchor Point’. Drag MergeCameraRight to ‘Right Camera’ (Vuforia will pop up a box saying ‘add vuforia components’ – click for both right and left cameras), then drag the MergeCameraLeft to ‘Left Camera’

On ARCamera property ‘Viewer’ select ‘Other Viewer’ and enter 1 for Viewer ID.

Save the Scene – now go ahead and build and run this scene on Android – you should now have a working AR app that can recognize the image stones marker and display the 3d mountain object. If you focus your gaze on the virtual ‘VR’ button for 2 seconds the app will transition you inside the full VR scene where you can look around, to exit the VR scene look straight down and focus on the ‘AR’ button for 2 seconds.

We now have a working AR/VR app – we need to make a few modifications to this scene to get it to run best in the MergeVR headset and to let us use the capactive touch buttons on the MergeVR headset to interact with the VR world.

The MergeVR headset needs the camera on the right side to work in AR mode with the Android, since the generic Vuforia demo doesn’t support this we have to make a few modifications to the MergeVR code to handle the change.

To move the viewport to the correct position when camera is on the right. Open the ‘MergeScreenManager.cs’ script in MergeVR->Scripts. In the function ‘SetViewPortResolutionAndPostion’ replace this line

viewportYpos = viewportBottom;

with

viewportYpos = viewportBottom+(Screen.height-viewportHeight);

and in the ‘MergeCameraController.cs’ script in MergeVR->Scripts comment out the following lines in the function AndroidGyroTracking

if (Input.deviceOrientation==DeviceOrientation.LandscapeRight || Input.deviceOrientation==DeviceOrientation.LandscapeLeft)

currentOrientation=Input.deviceOrientation; //only change on either full landscape

if (currentOrientation==DeviceOrientation.LandscapeRight) {

androidGyroRotation = new Quaternion (-Y, X, Z, -W);

androidGyroRotation *= Quaternion.Euler(180f,180f,0);

}

else

leaving

androidGyroRotation = new Quaternion (-Y, X, Z, -W); //default

Now build and run the app – the viewports will be aligned correctly and the VR scene will be working as well.

Now we need to add interaction with the VR scene (movement in this example) using the MergeVR capactive touch buttons on the headset.

Add new script called MergeEyeCustom to the MergeCameraController game object in your scene

using UnityEngine;

using System.Collections;

public class MergeEyeCustom : MonoBehaviour {

public float speed = 1.5f;

public float jumpSpeed = 10.0f;

public float gravity = 10.0f;

public bool allowJump = false;

Vector3 moveDirection = Vector3.zero;

float ydirection = 0f;

float xdirection = 0f;

// Use this for initialization

void Start () {

}

// Update is called once per frame

void Update () {

if (Merge.MSDK.isControllerConnected ()) {

ydirection = Merge.MSDK.getController ().GetAxis (“Vertical”);

xdirection = Merge.MSDK.getController ().GetAxis (“Horizontal”);

} else {

//Use arrow keys in editor – or touch capactive buttons if present in MergeVR headset

if (Input.GetKey (KeyCode.UpArrow) || Merge.MergeInput.GetInput(1))

ydirection = -1;

else if (Input.GetKey (KeyCode.DownArrow) || (!allowJump && Merge.MergeInput.GetInput(0)))

ydirection = 1;

else

ydirection = 0;

if (Input.GetKey (KeyCode.RightArrow))

xdirection = 1;

else if (Input.GetKey (KeyCode.LeftArrow))

xdirection = -1;

else

xdirection = 0;

}

moveDirection = new Vector3 (transform.forward.x * ydirection*speed, 0f, transform.forward.z * ydirection*speed);

moveDirection += new Vector3 (transform.right.x * xdirection*speed, 0f, transform.right.z * xdirection*speed);

transform.position+=moveDirection* Time.deltaTime;

}

Now you can use the left and right MergeVR headset buttons to move around the scene once you are in VR mode.

As an exercise you can do the following to polish the example.

remove or modify one of the gaze cursors

optimize rendering performance

add a rigid body or first person controller for more realistic collisions the VR scene

Using Parse SDK with Unity – running on an Android Device

Posted on June 29, 2015 by John Gillespie

If you are using Parse for your back-end server in a Unity app, and you want to deploy the app to Android, you may run into some issues with the built-in Parse plugin. Specifically (at least in my case with Unity 4.6 and Android KitKat) the Parse query object doesn’t ever return a result.

As a work-around I ended up going with a solution using Unity’s WWW class and the Parse REST API.


private IEnumerator GetParseDataForAndroid () {

	//new WWW object
	WWWForm form = new WWWForm();


	//set headers, not sure why but I had to set twice to get it to work
	Hashtable headers = form.headers.AddAuthorizationHeader("Your-Parse-Application-Id", "javascript-key=Your-Parse-REST-API-Key");

	headers.Add("X-Parse-Application-Id","Your-Parse-Application-Id");
	headers.Add("X-Parse-REST-API-Keyd","Your-Parse-REST-API-Key");

	WWW parseRequest = new WWW("https://Your-Parse-Application-Id:javascript-key=Your-Parse-REST-API-Key@api.parse.com/1/classes/yourObject", null, headers);


		//yield until request is done
		yield return parseRequest;


		//request is done, process results
		Debug.Log (" MergeVRBridge parse query eror result is " + parseRequest.error);


		//using simplejson
		var V = JSON.Parse (parseRequest.text);

		JSONArray results = V ["results"].AsArray;

		for (int i=0; i<results.Count; i++) {

			JSONClass jObject = results[i].AsObject;
			
			//do something interesting with results
			string sTitle = jObject["title"];
			Debug.Log(" yourObject " + sTitle);

			
			
		}




}

Android Studio and Unity3d

Posted on March 22, 2015 by John Gillespie

If you are building native Android plugins for Unity with Android Studio here are a couple of items that I struggled with. Note that all examples are using Unity 4.6 and Android Studio.

1. Unity Android plugin folder needs a jar file to function – but Android Studio build process does not produce a class jar file.

It turns out that Android Studio actually does create a jar file during the build process and stores it in an intermediate location (on the Mac it is in ‘build/intermediates/bundles/release/’).

Based on suggestions from here I created two new tasks in my gradle file – cleanjar and makejar – after a build I run both and now have the required jar file in my ‘build/outputs/’ folder.

task clearJar(type: Delete) {
delete ‘build/outputs/MergeVRBridge.jar’
delete ‘build/outputs/AndroidManifest.xml’
}

task makeJar(type: Copy) {
from(‘build/intermediates/bundles/release/’)
into(‘build/outputs/’)
include(‘classes.jar’)
include(‘AndroidManifest.xml’)
rename (‘classes.jar’, ‘MergeVRBridge.jar’)
}

2. If your Android library file requires a context – you can pass it to your lib from Unity.

AndroidJNI.AttachCurrentThread();
androidClass = new AndroidJavaClass(“com.mergelabs.MergeVR.MergeVRBridge“);

            using(AndroidJavaClass activityClass = new AndroidJavaClass(“com.unity3d.player.UnityPlayer“)) {

                activityContext = activityClass.GetStatic<AndroidJavaObject(“currentActivity“);
            }
            androidClass.CallStatic(“setContext“, activityContext);
            androidClass.CallStatic<int>(“mergeVRInit“);

and in your Android Java clase you’ll have the appropriate function

private static Context mContext;

static public void setContext(Context context) {

mContext = context;
}

3. You need bluetooth permissions set in your Android plugins manifest file.

If you just copy and paste the manifest file from your Android Studio build directory to the Unity Android plugin directory you will see compile and build errors. It appears in Unity 4.6 that if you place a manifest file in the Android plugin directory it overrides the Unity created manifest file for the entire app.

The best solution is to find the Unity generated manifest file, copy it to a temp directory and modify by adding any permissions you need (in my case bluetoth related), then add to the Android plugins folder – Unity will use this manifest file as the default and all will be well.

<?xml version=”1.0″ encoding=”utf-8″?>
<manifest xmlns:android=”http://schemas.android.com/apk/res/android” package=”com.merge.demo” android:versionName=”1.0″ android:versionCode=”1″ android:installLocation=”preferExternal”>

<uses-permission android:name=”android.permission.BLUETOOTH”/>
<uses-permission android:name=”android.permission.BLUETOOTH_ADMIN”/>
<uses-feature android:name=”android.hardware.bluetooth_le” android:required=”true”/>

<supports-screens android:smallScreens=”true” android:normalScreens=”true” android:largeScreens=”true” android:xlargeScreens=”true” android:anyDensity=”true” />

Unity3d and Parse.com error messages

Posted on August 13, 2014 by John Gillespie

I’ve used Parse.com quite a bit for iOS development and been very happy with it. Performance and scaling is good, and working with the cloud based service is very straight forward.

I decided to use Parse.com as the back end database solution for my latest Unity3d project. Parse now has an official Unity SDK that you can download from the Unity3d asset store.

Despite all of the benefits of using Parse with Unity there is one glaring problem. Because of the way the Unity implements their WWW class we can’t access response headers containing error information from Parse operations. So requests sent using the Parse SDK from Unity return responses which can be either successful, failed because of no connection, or failed because of some unknown cause.

Kalekindev has a great blog post addressing this issue. The solution is to create custom cloud code functions on Parse.com and call them instead. The cloud code handles the request and then returns either a success code or an appropriate error message.

Below is my initial cloud code running on Parse (cloud code is in javascript) to handle user signup.

Parse.Cloud.define("SignUp", function(request, response)
{
    var user = new Parse.User();
 
    user.set("username", request.params.username);
    user.set("password", request.params.password);
    user.set("email", request.params.email);
 
    user.signUp(null,
    {
        success: function(user)
        {
            response.success(
                {
                    "success": "Sign up successful."
                });
        },
        error: function(user, error)
        {
            // We must respond with a success in order to access the
            // result of the request inside Unity.
            response.success(
                {
                    "error": "Sign up failed.",
                    "code": error.code,
                    "message": error.message
                });
        }
    });
});

And here is the Unity3d C# code snipped that makes the call to the Parse.com cloud code and handles the return code appropriately if there is an error..

private IEnumerator CloudSignUp(string sUsername,string sPassword,string sEmail)
{
	var userInfo = new Dictionary
	{
		{"username", sUsername},
		{"password", sPassword},
		{"email", sEmail}


	};
	
	var t = ParseCloud.CallFunctionAsync<Dictionary>("SignUp", userInfo);
	
	while (!t.IsCompleted)
	{
		yield return null;
	}
	
	if (t.IsFaulted)
	{
		var parseException = (ParseException)t.Exception.InnerExceptions[0];
		
		if (parseException.Message.StartsWith("Could not resolve"))
		{
			Debug.Log("Failure: Connection error.");
			objStatus.text="Failure: Connection error.";
			objStatusPanel.SetActive(true);
		}
		else
		{
			// Possible mystery case?
			Debug.Log("Failure: Unknown cause.");
			objStatus.text="Failure: Unknown cause..";
			objStatusPanel.SetActive(true);
		}
	}
	else // Our request is successful but may still have failed server side.
	{
		object code;
		
		if (t.Result.TryGetValue("code", out code))
		{
			// Handle our call specific errors here.
			Debug.Log(string.Format("Failure: Code {0}", code));

			string sCode = string.Format("{0}", code);
	
			if (sCode.Equals("125")) {

				objStatus.text="Email is invalid";
				objStatusPanel.SetActive(true);

				Debug.Log("bad email");
			}

			 else if (sCode.Equals("202")) {
				
				objStatus.text="Username is already in use.";
				objStatusPanel.SetActive(true);
				
				Debug.Log("bad username");
			}

			else if (sCode.Equals("203")) {
				
				objStatus.text="Email is already in use.";
				objStatusPanel.SetActive(true);
				
				Debug.Log("bad username");
			}

			else {

				objStatus.text=string.Format("Failure: Code {0}", code);
				objStatusPanel.SetActive(true);
	
			}


		}
		else
		{
			Debug.Log("Success!");
		}
	}
}

This approach can be used for all Parse.com functions called from Unity and gives us a robust way to handle exceptions and errors.

Product7

Real Value Delivered Weekly, Guaranteed

Tag Archives: Unity3d

Vuforia / MergeVR Integration

Using Parse SDK with Unity – running on an Android Device

Android Studio and Unity3d

Unity3d and Parse.com error messages