React Native for Mobile – Initial Impressions

nativesmall

I’ve always done mobile development in the device native language (Objective-C/Swift for iOS, and Java for Android)

Recently I worked on a project that needed to be deployed on Android, iOS and eventually the web, using the same code base if possible. After reviewing the options I decided to use React Native.

React Native lets you build mobile apps using only JavaScript. It uses the same design as React, letting you compose a rich mobile UI from declarative components. It is the library used to build Facebook, Instagram, and many other applications.

The project was originally a native Android app – used by TED for handling attendees checking in to individual sessions during a conference. It makes multiple API calls to the server back-end, does NFC scanning, needs local data storage, and must work offline as well as online.

I’ll post more further down the road about working in React Native so for now I’ll list some initial pros and cons…

Pros

Performance – better and faster json parsing, cleaner program flow, screen redraw update appears faster than native if state is used properly

Multi-platform – same code base for Android and iOS, can also develop for web using React Native for Web.

Open source – strongy supported by facebook, code available on github, 1,700 contributors, 68,000 stars

Cons

Javascript – the language still makes me nervous, I like a strongly typed language – recommend using typescript or flow to lower your risk.

Debugging – debugging tools aren’t as mature as I would like, using Chrome and a lot of console.log calls – which is definitely old school.

Tools – no integrated tools, I use a mixture of the command line, Chrome, and Sublime Text as my editor.

<>

Further reading

How to Get Started with React Native

The Basics of React Native

An iOS Developer’s Journey Into React Native

Building the F8 App

Vuforia / MergeVR Integration

This post outlines the steps required to integrate Vuforia for Digital Eyeware with the MergeVR SDK in Unity. The result of this integration will be an Augmented Reality demo app that can be run in the MergeVR headset on your Android device. It will recognize an image marker and display a 3d object on top of that marker, and allow the user to trigger a virtual button on the object – then enter in VR mode and move around the VR scene using the MergeVR headset capactive input buttons.

https://developer.qualcomm.com/software/vuforia-augmented-reality-sdk

http://mergevr.com/

 

Requirements

Android mobile device

Unity3d – version 4.6.*

Vuforia – Download the core Vuforia unity package – vuforia-unity-5-0-5 (you must be a registered Vuforia developer before the download) and the eyeware samples vuforia-samples-eyewear-unity-5-0-5

MergeVR – download latest Merge SDK Unity package- MergeVR_version_v062

 

Steps

 

  • Create a new empty Unity Project (4.6.*)
  • Import the Merge SDK Unity package (MergeVR_version_v062.unitypackage)
  • Import the Vuforia core Unity package (vuforia-unity-5-0-5.unitypackage)
  • Import the Vuforia Digital Eyeware sample AR/VR package from the vuforia-samples-eyewear-unity-5-0-5.zip file (arvr-5-0-5.unitypackage)
  • In the Scenes folder find the Scene ‘Vuforia-3-AR-VR’ – Duplicate it (Edit->Duplicate) and rename the new scene ‘DemoVRAR’
  • Open ‘DemoVRAR’ scene.

 

 

 

 

  • Stop running the scene (if you haven’t already)

 

  • Find the MergeVR -> Prefabs folder. Drag the MergeCameraController and MergeSDK prefabs into the scene Hierarchy.

 

  • Expand the MergeCameraController tree.

 

  • Click on ARCamera under UserHead and view the Inspector Pane. Check (enable) the ‘Bind Alternate Camera’ property.

 

  • Drag the MergeCameralController root transform to the ARCamera property ‘Central Anchor Point’. Drag MergeCameraRight to ‘Right Camera’ (Vuforia will pop up a box saying ‘add vuforia components’ – click for both right and left cameras), then drag the MergeCameraLeft to ‘Left Camera’

 

  • On ARCamera property ‘Viewer’ select ‘Other Viewer’ and enter 1 for Viewer ID.

 

  • Save the Scene – now go ahead and build and run this scene on Android – you should now have a working AR app that can recognize the image stones marker and display the 3d mountain object. If you focus your gaze on the virtual ‘VR’ button for 2 seconds the app will transition you inside the full VR scene where you can look around, to exit the VR scene look straight down and focus on the ‘AR’ button for 2 seconds.

 

  • We now have a working AR/VR app – we need to make a few modifications to this scene to get it to run best in the MergeVR headset and to let us use the capactive touch buttons on the MergeVR headset to interact with the VR world.

 

  • The MergeVR headset needs the camera on the right side to work in AR mode with the Android, since the generic Vuforia demo doesn’t support this we have to make a few modifications to the MergeVR code to handle the change.

 

  • To move the viewport to the correct position when camera is on the right. Open the ‘MergeScreenManager.cs’ script in MergeVR->Scripts. In the function ‘SetViewPortResolutionAndPostion’ replace this line

 

viewportYpos = viewportBottom;

 

with

 

viewportYpos = viewportBottom+(Screen.height-viewportHeight);

 

and in the ‘MergeCameraController.cs’ script in MergeVR->Scripts comment out the following lines in the function AndroidGyroTracking

 

/*

if (Input.deviceOrientation==DeviceOrientation.LandscapeRight || Input.deviceOrientation==DeviceOrientation.LandscapeLeft)

currentOrientation=Input.deviceOrientation; //only change on either full landscape

 

if (currentOrientation==DeviceOrientation.LandscapeRight) {

androidGyroRotation = new Quaternion (-Y, X, Z, -W);

androidGyroRotation *= Quaternion.Euler(180f,180f,0);

}

else

*/

 

leaving

 

androidGyroRotation = new Quaternion (-Y, X, Z, -W); //default

 

  • Now build and run the app – the viewports will be aligned correctly and the VR scene will be working as well.

 

  • Now we need to add interaction with the VR scene (movement in this example) using the MergeVR capactive touch buttons on the headset.

 

  • Add new script called MergeEyeCustom to the MergeCameraController game object in your scene

 

using UnityEngine;

using System.Collections;

 

public class MergeEyeCustom : MonoBehaviour {

 

public float speed = 1.5f;

public float jumpSpeed = 10.0f;

public float gravity = 10.0f;

public bool allowJump = false;

 

Vector3 moveDirection = Vector3.zero;

float ydirection = 0f;

float xdirection = 0f;

 

// Use this for initialization

void Start () {

 

}

 

// Update is called once per frame

void Update () {

 

if (Merge.MSDK.isControllerConnected ()) {

ydirection = Merge.MSDK.getController ().GetAxis (“Vertical”);

xdirection = Merge.MSDK.getController ().GetAxis (“Horizontal”);

} else {

//Use arrow keys in editor – or touch capactive buttons if present in MergeVR headset

if (Input.GetKey (KeyCode.UpArrow) || Merge.MergeInput.GetInput(1))

ydirection = -1;

else if (Input.GetKey (KeyCode.DownArrow) || (!allowJump && Merge.MergeInput.GetInput(0)))

ydirection = 1;

else

ydirection = 0;

 

if (Input.GetKey (KeyCode.RightArrow))

xdirection = 1;

else if (Input.GetKey (KeyCode.LeftArrow))

xdirection = -1;

else

xdirection = 0;

}

 

moveDirection = new Vector3 (transform.forward.x * ydirection*speed, 0f, transform.forward.z * ydirection*speed);

moveDirection += new Vector3 (transform.right.x * xdirection*speed, 0f, transform.right.z * xdirection*speed);

 

 

transform.position+=moveDirection* Time.deltaTime;

 

}

}

 

  • Now you can use the left and right MergeVR headset buttons to move around the scene once you are in VR mode.

 

  • As an exercise you can do the following to polish the example.

 

remove or modify one of the gaze cursors

 

optimize rendering performance

 

add a rigid body or first person controller for more realistic collisions the VR scene

Using Parse SDK with Unity – running on an Android Device

unityandroid

If you are using Parse for your back-end server in a Unity app, and you want to deploy the app to Android, you may run into some issues with the built-in Parse plugin. Specifically (at least in my case with Unity 4.6 and Android KitKat) the Parse query object doesn’t ever return a result.

As a work-around I ended up going with a solution using Unity’s WWW class and the Parse REST API.

 

 

 


private IEnumerator GetParseDataForAndroid () {

	//new WWW object
	WWWForm form = new WWWForm();


	//set headers, not sure why but I had to set twice to get it to work
	Hashtable headers = form.headers.AddAuthorizationHeader("Your-Parse-Application-Id", "javascript-key=Your-Parse-REST-API-Key");

	headers.Add("X-Parse-Application-Id","Your-Parse-Application-Id");
	headers.Add("X-Parse-REST-API-Keyd","Your-Parse-REST-API-Key");

	WWW parseRequest = new WWW("https://Your-Parse-Application-Id:javascript-key=Your-Parse-REST-API-Key@api.parse.com/1/classes/yourObject", null, headers);


		//yield until request is done
		yield return parseRequest;


		//request is done, process results
		Debug.Log (" MergeVRBridge parse query eror result is " + parseRequest.error);


		//using simplejson
		var V = JSON.Parse (parseRequest.text);

		JSONArray results = V ["results"].AsArray;

		for (int i=0; i<results.Count; i++) {

			JSONClass jObject = results[i].AsObject;
			
			//do something interesting with results
			string sTitle = jObject["title"];
			Debug.Log(" yourObject " + sTitle);

			
			
		}




}


Android Studio and Unity3d

If you are building native Android plugins for Unity with Android Studio here are a couple of items that I struggled with. Note that all examples are using Unity 4.6 and Android Studio.

1. Unity Android plugin folder needs a jar file to function – but Android Studio build process does not produce a class jar file.

It turns out that Android Studio actually does create a jar file during the build process and stores it in an intermediate location (on the Mac it is in ‘build/intermediates/bundles/release/’).

Based on suggestions from here I created two new tasks in my gradle file – cleanjar and makejar – after a build I run both and now have the required jar file in my ‘build/outputs/’ folder.

 task clearJar(type: Delete) {
delete ‘build/outputs/MergeVRBridge.jar’
delete ‘build/outputs/AndroidManifest.xml’
}

task makeJar(type: Copy) {
from(‘build/intermediates/bundles/release/’)
into(‘build/outputs/’)
include(‘classes.jar’)
include(‘AndroidManifest.xml’)
rename (‘classes.jar’, ‘MergeVRBridge.jar’)
}

2. If your Android library file requires a context – you can pass it to your lib from Unity.

AndroidJNI.AttachCurrentThread();
           androidClass = new AndroidJavaClass(com.mergelabs.MergeVR.MergeVRBridge);

            using(AndroidJavaClass activityClass = new AndroidJavaClass(com.unity3d.player.UnityPlayer)) {
              
                activityContext = activityClass.GetStatic<AndroidJavaObject(currentActivity);
            }
            androidClass.CallStatic(setContextactivityContext);
            androidClass.CallStatic<int>(mergeVRInit);

and in your Android Java clase you’ll have the appropriate function

private static Context mContext;

 static public void setContext(Context context) {

mContext = context;
}

3. You need bluetooth permissions set in your Android plugins manifest file.

If you just copy and paste the manifest file from your Android Studio build directory to the Unity Android plugin directory you will see compile and build errors. It appears in Unity 4.6 that if you place a manifest file in the Android plugin directory it overrides the Unity created manifest file for the entire app.

The best solution is to find the Unity generated manifest file, copy it to a temp directory and modify by adding any permissions you need (in my case bluetoth related), then add to the Android plugins folder – Unity will use this manifest file as the default and all will be well.

<?xml version=”1.0″ encoding=”utf-8″?>
<manifest xmlns:android=”http://schemas.android.com/apk/res/android&#8221; package=”com.merge.demo” android:versionName=”1.0″ android:versionCode=”1″ android:installLocation=”preferExternal”>

<uses-permission android:name=”android.permission.BLUETOOTH”/>
<uses-permission android:name=”android.permission.BLUETOOTH_ADMIN”/>
<uses-feature android:name=”android.hardware.bluetooth_le” android:required=”true”/>

<supports-screens android:smallScreens=”true” android:normalScreens=”true” android:largeScreens=”true” android:xlargeScreens=”true” android:anyDensity=”true” />

<application android:theme=”@android:style/Theme.NoTitleBar” android:icon=”@drawable/app_icon” android:label=”@string/app_name” android:debuggable=”false”>
<activity android:name=”com.unity3d.player.UnityPlayerNativeActivity” android:label=”@string/app_name” android:screenOrientation=”sensorLandscape” android:launchMode=”singleTask” android:configChanges=”mcc|mnc|locale|touchscreen|keyboard|keyboardHidden|navigation|orientation|screenLayout|uiMode|screenSize|smallestScreenSize|fontScale”>
<intent-filter>
<action android:name=”android.intent.action.MAIN” />
<category android:name=”android.intent.category.LAUNCHER” />
<category android:name=”android.intent.category.LEANBACK_LAUNCHER” />
</intent-filter>
<meta-data android:name=”unityplayer.UnityActivity” android:value=”true” />
<meta-data android:name=”unityplayer.ForwardNativeEventsToDalvik” android:value=”false” />
</activity>
</application>
<uses-sdk android:minSdkVersion=”18″ android:targetSdkVersion=”21″ />
<uses-feature android:glEsVersion=”0x00020000″ />
<uses-feature android:name=”android.hardware.touchscreen” android:required=”false” />
<uses-feature android:name=”android.hardware.touchscreen.multitouch” android:required=”false” />
<uses-feature android:name=”android.hardware.touchscreen.multitouch.distinct” android:required=”false” />
<uses-permission android:name=”android.permission.WAKE_LOCK” />
</manifest>

 

 

 

 

 

 

 

 

Using SourceTree on the Mac with github

Screen Shot 2015-02-19 at 4.40.17 PM

I’m using github for a large project with multiple developers – there have been a few wrinkles using the github mac client so we switched to SourceTree – a free github client. We’re using it to implement gitflow – I’ve been very happy with it so far – below is our process…

<>

Before you start a new feature

  • make sure you have the latest copy of the dev branch (do a fetch on remote, then pull if dev branch shows any changes)
  • use gitflow command ‘new feature’ and name it
  • make your changes to code
  • stage files, then commit and push all changes to your feature branch
  • before finishing feature (merging with development) – use fetch to check status of development branch,
    • if there have been commits by other members of the team since you downloaded the development branch you will need to merge those changes to your feature branch before finishing the feature
      • checkout the development branch, then pull it to get current
      • re-checkout your new feature branch
      • use gitflow ‘finish current’ (set to ‘delete branch’ and ‘force delete’)
      • merge conflict message, go to working copy, right click resolve conflicts – launch external merge tool, choose resolution, file save merge, quit merge tool (do not quit merge tool until you’re done with all merges
      • commit and push merge results
      • now push the new dev branch
    • if there are no merge issues then just push the new dev branch up to the remote origin
  • push develop changes up to remote (if not done already)
  • check github site to confirm your changes are present (just until you are comfortable with this process)

Thinking of You

thinkshot

Just launched a fun app for Mason Software Company called Thinking of You – you can follow it on facebook as well.

The app uses the iOS contact list extensively to send ‘Thinking of You’ push and sms messages. Apple requires that you request permission from the user before you can access the address book, below is a code snippet showing how that is done…

 

 


#pragma mark -
#pragma mark Address Book Access

// Check the authorization status of our application for Address Book
-(void)checkAddressBookAccess
{
    switch (ABAddressBookGetAuthorizationStatus())
    {
            // Update our UI if the user has granted access to their Contacts
        case  kABAuthorizationStatusAuthorized:
            [self accessGrantedForAddressBook];
            break;
            // Prompt the user for access to Contacts if there is no definitive answer
        case  kABAuthorizationStatusNotDetermined :
            [self requestAddressBookAccess];
            break;
            // Display a message if the user has denied or restricted access to Contacts
        case  kABAuthorizationStatusDenied:
        case  kABAuthorizationStatusRestricted:
        {
            UIAlertView *alert = [[UIAlertView alloc] initWithTitle:@"Privacy Warning"
                                                            message:@"Permission was not granted for Contacts."
                                                           delegate:nil
                                                  cancelButtonTitle:@"OK"
                                                  otherButtonTitles:nil];
            [alert show];
        }
            break;
        default:
            break;
    }
}

// Prompt the user for access to their Address Book data
-(void)requestAddressBookAccess
{
    ViewController * __weak weakSelf = self;

    ABAddressBookRequestAccessWithCompletion(self.addressBook, ^(bool granted, CFErrorRef error)
                                             {
                                                 if (granted)
                                                 {

                                                     dispatch_async(dispatch_get_main_queue(), ^{
                                                         [weakSelf accessGrantedForAddressBook];

                                                     });
                                                 }
                                             });
}

// This method is called when the user has granted access to their address book data.
-(void)accessGrantedForAddressBook
{
    //do something interesting
}

Unity3d and Parse.com error messages

I’ve used Parse.com quite a bit for iOS development and been very happy with it. Performance and scaling is good, and working with the cloud based service is very straight forward.

I decided to use Parse.com as the back end database solution for my latest Unity3d project. Parse now has an official Unity SDK that you can download from the Unity3d asset store.

Despite all of the benefits of using Parse with Unity there is one glaring problem. Because of the way the Unity implements their WWW class we can’t access response headers containing error information from Parse operations.  So requests sent using the Parse SDK from Unity return responses which can be either successful, failed because of no connection, or failed because of some unknown cause.

Kalekindev has a great blog post addressing this issue. The solution is to create custom cloud code functions on Parse.com and call them instead. The cloud code handles the request and then returns either a success code or an appropriate error message.

Below is my initial cloud code running on Parse (cloud code is in javascript) to handle user signup.

Parse.Cloud.define("SignUp", function(request, response)
{
    var user = new Parse.User();
 
    user.set("username", request.params.username);
    user.set("password", request.params.password);
    user.set("email", request.params.email);
 
    user.signUp(null,
    {
        success: function(user)
        {
            response.success(
                {
                    "success": "Sign up successful."
                });
        },
        error: function(user, error)
        {
            // We must respond with a success in order to access the
            // result of the request inside Unity.
            response.success(
                {
                    "error": "Sign up failed.",
                    "code": error.code,
                    "message": error.message
                });
        }
    });
});

 

And here is the Unity3d C# code snipped that makes the call to the Parse.com cloud code and handles the return code appropriately if there is an error..

 

private IEnumerator CloudSignUp(string sUsername,string sPassword,string sEmail)
{
	var userInfo = new Dictionary
	{
		{"username", sUsername},
		{"password", sPassword},
		{"email", sEmail}


	};
	
	var t = ParseCloud.CallFunctionAsync<Dictionary>("SignUp", userInfo);
	
	while (!t.IsCompleted)
	{
		yield return null;
	}
	
	if (t.IsFaulted)
	{
		var parseException = (ParseException)t.Exception.InnerExceptions[0];
		
		if (parseException.Message.StartsWith("Could not resolve"))
		{
			Debug.Log("Failure: Connection error.");
			objStatus.text="Failure: Connection error.";
			objStatusPanel.SetActive(true);
		}
		else
		{
			// Possible mystery case?
			Debug.Log("Failure: Unknown cause.");
			objStatus.text="Failure: Unknown cause..";
			objStatusPanel.SetActive(true);
		}
	}
	else // Our request is successful but may still have failed server side.
	{
		object code;
		
		if (t.Result.TryGetValue("code", out code))
		{
			// Handle our call specific errors here.
			Debug.Log(string.Format("Failure: Code {0}", code));

			string sCode = string.Format("{0}", code);
	
			if (sCode.Equals("125")) {

				objStatus.text="Email is invalid";
				objStatusPanel.SetActive(true);

				Debug.Log("bad email");
			}

			 else if (sCode.Equals("202")) {
				
				objStatus.text="Username is already in use.";
				objStatusPanel.SetActive(true);
				
				Debug.Log("bad username");
			}

			else if (sCode.Equals("203")) {
				
				objStatus.text="Email is already in use.";
				objStatusPanel.SetActive(true);
				
				Debug.Log("bad username");
			}

			else {

				objStatus.text=string.Format("Failure: Code {0}", code);
				objStatusPanel.SetActive(true);
	
			}


		}
		else
		{
			Debug.Log("Success!");
		}
	}
}

 

This approach can be used for all Parse.com functions called from Unity and gives us a robust way to handle exceptions and errors.

Augmented Reality on iOS

Durovis Dive

I’ve been working on an AR (augmented reality) application for iOS and thought I would share some of the available software libraries and devices for iOS.

Hardware

Although an iPhone is not as ‘hands free’ as a dedicated AR device (Moverio, Vuzix, or the Meta SpaceGlasses) there are interesting hardware solutions to use your iPhone or Android smart phone as a AR client…

Durovis Dive

Durovis Dive

The Durovis Dive works with any Android or iOS smartphone featuring a gyroscope and an accelerometer and a display not larger than 5 inches. You just insert your smartphone, start the application and adjust the lenses to your eyes.

FOV2GO

FOV2GO

You can construct your own FOV2GO Model D Viewer – for the iPhone 4/4S, the Samsung Galaxy Note, or most Android smart phones – out of foam board and a couple of plastic lenses. Just download the instructions and the appropriate template.

Software

Source for my initial research and a great resource at Augmented Reality SDK Comparison

 

Open Source and/or Free

Argon
iOs – augmented reality from Georgi Tech – appears to be open source but I can’t find the code available for download – based on KHARMA – a KML/HTML Augmented Reality Mobile Architecture

iPhone ARKit
iOS – iPhone ARKit is a small set of class that can give you augmented reality in any iPhone application.

iPhone Augmented Reality Toolkit
This version of the iPhone ARKit is a forked version of the ARKit started on GitHub by Zac White.

mixare – Open Source Augmented Reality Engine
iOS & Android – It works as a completely autonomous application and is available as well for the development of own implementations.

PRAugmentedReality
open source – Augmented Reality Framework for iOS

Vuforia
(from Qualcomm)
iOS, Android, and Unity 3D
Very powerful tracker based AR SDK – free for commercial development

 

Proprietary and/or Paid

ARPA
iOS & Android

ARToolKit
iOS & Android – a very powerful Lib with a lot of work being done with it – has an open source version (but not on mobile devices)

Catchoom
iOS & Android

Metaio
iOS & Android (includes Junaio) – lots of products, including content creation and a cloud offering

Robocortex

String
Comes in vanilla OpenGL and Unity flavours. Add AR to any iOS project, regardless of 3D engine.

Wikitude
iOS, Android and BlackBerry 10 – both tools for client and creation – also has an AR browser app – very popular with a lot of dev shops using it

Xloudia
cloud based AR recognition and tracking

 

Roll your Own

Augmented Reality iOS Tutorial: Location Based

Augmented Reality iOS Tutorial: Marker Tracking

Results of Hackathon

angle hack austin

preparing for demos

This weekend I won 2nd place at the AppHack Austin with ShareMesh – an iOS based encrypted peer to peer messaging app that does not require an internet connection to operate. The app used the new iOS 7 features iBeacon and Multipeer Connectivity to handle discovery and communication between devices.

I had a great time and there were a lot of interesting projects and very talented people working on them (see a list at hackathon.io).

The Capital Factory facilities are world class and the folks from AngelHack and our sponsors were great hosts. Looking forward for the chance to go to the Spring 2014 HACKcelerator program in San Francisco.

hackathon_nov_2014_codingonsaturday hackathon_nov_2013_presentation