[ad_1]
Introduction
Nex Gen Media Server is best known as a multi-purpose media streaming server to deliver live and stored video to a variety of devices. The same media server can be embedded into a mobile application to facilitate real time video communication. Here we will use the NGMS API to facilitate building a video chat client for Android devices using the Android Software Developer Kit (SDK) and Google Android Native Developer Kit (NDK).
We will use NGMS to provide all of the network transport RTP services, video frame compression and decompression. NGMS will run as a natively compiled shared object used by our Android app. We will use a Java Native Interface (JNI) bridging layer to invoke native methods from Java application code.
Pre-requisistes
This article assumes that you have a basic understanding and at least an intermediate level of experience building Android apps. I will not go over the details of setting up and Android project in an iDE such as Eclipse. If you have previously created Android apps, or at least went through some example tutorials you should be able to utilize this tutorial to create a real-time video streaming client.
If you haven't already done so, you will need to download the Google Android SDK (Software Developer Kit). In this example I used Revision 15 on a Linux 32bit machine. The java application layer is used to create the actual Android app. The app code will interface with a JNI layer to invoke the NGMS API routines.
You will also need to download the Google Android NDK (Native Developer Kit). In this example I used Revision 6b on a Linux 32bit machine. The NDK will be used for creating the native interface layer which serves as the glue between the native code and Java application code. You should also have a basic understand of the C programming language.
You should first begin by creating a skeleton Android application using an IDE such as Eclipse. In this example, the application will be called ngmsclient. The Java package name will be called com.example.ngmsclient and will target Android OS 2.3 or higher. The root project directory should contain a folder called “jni” which will contain the native sources used to interface with the Java application code. The example below assumes that you have a skeleton Android app working and will show you how to integrate an ExampleChat class into your Android project.
The Native Layer
The NGMS core embedded library is written in C and is packaged as a shared object file. Since the Android OS is based on Linux, the NGMS core library will run natively within your Android application space. You will need to obtain the NGMS core library for Android OS from the ngmsvid.com website. The essential components are the library files libngms.so, libxcode.so, and the header file ngmslib.h. The ngms bundled.so files will be packaged along with your ngmsclient.apk to provide video chat services.
The JNI Layer
We will build a shared library called ngmsglue.so which will serve as the glue between the ngms API and our Java application code. The structure of the Java Native Interface (JNI) directory within your ngmsclient project directory should look like this.
jni/Android.mk
jni/ngmsglue.c
jni/ngms/include/ngmslib.h
jni/ngms/lib/libngms.so
jni/ngms/lib/libxcode.so
ngmsglue.c will contain the code to control NGMS output and input features. The code will maintain two seperate NGMS API contexts, one for stream output parameters and one for stream input parameters. You should notice that the naming convention of each function should match the Java package and class name from where you are invoking the native code.
ngmsStartReceiver is called to initialize the NGMS capture input API to listen on port 5004 for a network video stream encapsulated over MPEG-2 TS. Each successive video frame will be automatically demuxed and decoded into RGB565 pixel format. The complete frame can then be read by calling ngmsReceiveFrame.
ngmsStartSender is called to initialize the NGMS stream output API to output encoded video frames to a remote host over port 5004. ngmsTransmitFrame is called to encode and transmit a single raw frame in NV21 pixel format.
/*
* ngmsglue.c
*
* JNI layer to access the NGMS streaming and capture API
*
*/#include
#include
#include
#include
#include "ngmslib.h"/**
* Holds the configuration for the NGMS RTP receiver and decoder
*/
static NGMSLIB_STREAM_PARAMS_T ngmsReceiver;/**
* Holds the configuration for the NGMS RTP sender and encoder
*/
static NGMSLIB_STREAM_PARAMS_T ngmsSender;/**
* Starts the NGMS capture service
*/
jint Java_com_example_ngmsclient_ExampleChat_ngmsStartReceiver (JNIEnv* _env, jobject _thiz)
{char xcoderConfig[512];
char sdpConfig[512];
int i;/*
* Before doing anything call ngmslib_open to open the NGMS service
*/
i = ngmslib_open(&ngmsReceiver);
if (i!= NGMS_RC_OK)
{
__android_log_print(ANDROID_LOG_ERROR, "ngms", "ngmslib_open failed with error %d", i);
return -1;
}/*
* Construct an xcoder configuration key value string
*/
sprintf(xcoderConfig, "vc=rgb565,vx=320,vy=240");/*
* Construct an SDP configuration for the RTP receiver
* Here we setup NGMS to read data over a MPEG-2 Transport Stream.
*/
sprintf(sdpConfig, "sdp://"
"m=video 5004 RTP/AVP 33\n"
"a=rtpmap:33 MP2T/90000\n"
"a=fmtp:33\n");/*
* Customize the capture configuration parameters
*/
ngmsReceiver.inputs[0] = sdpConfig;
ngmsReceiver.strxcode = xcoderConfig;
ngmsReceiver.islive = 1;
ngmsReceiver.noaud = 1;/*
* Start the RTP receiver
*/
i = ngmslib_stream(&ngmsReceiver);if (i!= NGMS_RC_OK)
{
__android_log_print(ANDROID_LOG_ERROR, "ngms", "ngmslib_stream failed with error %d", i);
ngmslib_close(&ngmsReceiver);
return -1;
}return 0;
}/**
* Stops the NGMS RTP capture service
*/
jint Java_com_example_ngmsclient_ExampleChat_ngmsStopReceiver (JNIEnv* _env, jobject _thiz)
{/*
* Stop the RTP receiver
*/
ngmslib_close(&ngmsReceiver);return 0;
}/**
* NGMS Callback operation which blocks until a complete video frame has been received and decoded
*/
jint Java_com_example_ngmsclient_ExampleChat_ngmsReceiveFrame (JNIEnv* _env, jobject _thiz, jbyteArray frameBytes)
{
jstring str;jboolean copy;
jbyte* buffer = (*_env)->GetByteArrayElements(_env, frameBytes, ©);
jsize max = (*_env)->GetArrayLength(_env, frameBytes);
int frameSize;frameSize = ngmslib_readVidFrame(&ngmsReceiver, buffer, max, NULL);
(*_env)->ReleaseByteArrayElements(_env, frameBytes, buffer, JNI_ABORT);
return frameSize;
}
/**
* Starts the NGMS streaming service
*/
jint Java_com_example_ngmsclient_ExampleChat_ngmsStartSender (JNIEnv* _env, jobject _thiz, jstring remoteAddress)
{char xcoderConfig[512];
char filter[512];
char destination[512];
const char *p;
int i;/*
* Before doing anything call ngmslib_open to open the NGMS service
*/
i = ngmslib_open(&ngmsSender);if (i!= NGMS_RC_OK)
{
__android_log_print(ANDROID_LOG_ERROR, "ngms", "ngmslib_open failed with error %d", i);
return -1;
}/*
* Get the destination host string
*/
p = (*_env)->GetStringUTFChars(_env, remoteAddress, 0);sprintf(destination, "rtp://%s:5004", p);
(*_env)->ReleaseStringUTFChars(_env, remoteAddress, p);
/*
* Customize the stream configuration parameters
*/
ngmsSender.output = destination;
sprintf(xcoderConfig, "vc=h264,vp=66,vb=250,vx=320,vy=240,vgmax=2000,vgmin=1500,vfr=15,vcfrout=-1,vth=1,vl=1,vsc=1,vf=2");
ngmsSender.strxcode = xcoderConfig;
sprintf(filter, "type=yuv420sp, ngmsSender.strfilters[0] = filter;
ngmsSender.inputs[0]= "/dev/dummyvideo";
ngmsSender.noaud = 1;/*
* Start the RTP sender
*/
i = ngmslib_stream(&ngmsSender);if (i!= NGMS_RC_OK)
{
__android_log_print(ANDROID_LOG_ERROR, "ngms", "ngmslib_stream failed with error %d", i);
ngmslib_close(&ngmsSender);
return -1;
}return 0;
}/**
* Stops the NGMS streaming service
*/
jint Java_com_example_ngmsclient_ExampleChat_ngmsStopSender (JNIEnv* _env, jobject _thiz)
{/*
* Stop the RTP sender
*/
ngmslib_close(&ngmsSender);return 0;
}/**
* Encodes and transmits a single video frame using local RTP transport
*/
jint Java_com_example_ngmsclient_ExampleChat_ngmsTransmitFrame (JNIEnv* _env, jobject _thiz, jbyteArray frameBytes)
{jboolean copy;
jbyte* buffer = (*_env)->GetByteArrayElements(_env, frameBytes, ©);
jsize size = (*_env)->GetArrayLength(_env, frameBytes);
int i;i = ngmslib_onVidFrame(&ngmsSender, buffer, size);
(*_env)->ReleaseByteArrayElements(_env, frameBytes, buffer, JNI_ABORT);
return i;
}
We will create the makefile Android.mk which is used by the Android NDK to build our ngmsglue library.
#Example Android.mk MakefileLOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE:= ngmsglue
LOCAL_SRC_FILES:= ngmsglue.c
LOCAL_C_INCLUDES:= ngms/include
LOCAL_LDFLAGS:= -Lngms/lib
LOCAL_LDLIBS:= -llog -lm -lngms -lxcodeinclude $(BUILD_SHARED_LIBRARY)
To build the ngmsglue.so shared library simply do the following.
cd [YOUR NGMSCLIENT PROJECT DIR]/jni
${ANDROID_NDK_HOME}/ndk-build
This should produce the output file [YOUR NGMSCLIENT PROJECT DIR]/libs/armeabi/ibngmsglue.so
You may need to manually copy the libraries ngms/lib/libngms.so and ngms/lib/libxcode.so into the libs/armeabi output directory because the contents of this directory will be automatically packaged into the project.apk application.
The Java Application Layer
The java application layer consists of the application level logic for the Android app we are building. To be able to utilize the native code from the previous section we will need to create a simple application using the Android SDK. This example demonstrates using the ExampleChat class which can be used by your own application to provide real-time interactive video streaming. It does not show all the steps necessary to create a complete Android video streaming application. An intermediate level Android developer should be able to complete the task by building upon and including the ExampleChat class in their full-blown application.
The ExampleChat class shown below is used to control the NGMS stream sender and the NGMS stream receiver. The method startSender can be called to begin streaming video from the native camera using the camera preview mechanism. Each video frame output by the camera is handed down to the NGMS API where it will be encoded and transmitted to the remote party using native RTP encapsulation. The method starReceiver can be called to begin receiving video from a remote instance of the same app. startReceiver launches a thread which performs a blocking poll for the next available received and decoded video frame. Together, these two methods can be used to establish a bi-directional video stream between two remote instances of the same app across a network.
/*
* ExampleChat.java
*
*/package com.example.ngmsclient;
import android.hardware.Camera;
import android.hardware.Camera.Parameters;
import android.hardware.Camera.PreviewCallback;
import android.graphics.ImageFormat;public class ExampleChat
{
private boolean myReceiverRunning;
private Camera myCamera;
private PreviewCallback myCameraCallback;/**
* Change this value to the destination where the video will be streamed
*/
private static final DESTINATION = "10.0.0.1";/**
* method to initialize and start sending video from the local camera
*/
public void startSender () throws Exception
{myCamera = Camera.open();
Parameters parameters = myCamera.getParameters();
parameters.setPreviewFormat(ImageFormat.NV21);
parameters.setPreviewFrameRate(15);
parameters.setPreviewSize(320, 240);myCamera.setParameters(parameters);
/**
* Call the NGMS native method to perform streaming initialization
*/
ngmsStartSender(DESTINATION);myCameraCallback = new PreviewCallback()
{
public void onPreviewFrame(byte[] frameBytes, Camera camera)
{
/**
* Call the NGMS native routine to encode and transmit the video frame
* to the remote destination using RTP transport.
*/
ngmsTransmitFrame(frameBytes);
}
}myCamera.startPreview();
myCamera.setPreviewCallback(myCameraCallback);}
/**
* method to stop sending video
*/
public void stopSender () throws Exception
{
ngmsStopSender();
}/**
* method to start receiving remote video
*/
public void startReceiver () throws Exception
{/**
* Call the NGMS native method to perform input initialization
*/
ngmsStartReceiver();/**
* Start the thread which will poll for received video frames
*/
new Thread(myVideoFrameReceiver).start();}
/**
* method to stop receiving remote video
*/
public void stopReceiver () throws Exception
{
myReceiverRunning = false;
ngmsStopReceiver();
}/**
* Thread used to check for decoded video frames
* received by the NGMS native layer
*/
private Runnable myVideoFrameReceiver = new Runnable()
{
public void run()
{
/**
* Allocate a buffer to hold a video frame 320 x 240 pixels
* in RGB565 format, taking 16 bits (2 bytes) per pixel.
*/
byte[] frameBytes = new byte[ 320 * 240 * 2 ];
int size;
myReceiverRunning = true;while(myReceiverRunning)
{
/**
* Block until a frame has been received and decoded by the native layer
*/
size = ngmsReceiveFrame(frameBytes);
if(size > 0)
{
/**
* The frameBytes array represents a video frame in RGB565 format.
* These bytes can be converted into a Bitmap object for display
* on the screen.
*/
}
}
}
}/**
* Static initialization to load native libraries into address space
*/
static
{
System.loadLibrary("xcode");
System.loadLibrary("ngms");
System.loadLibrary("ngmsglue");}
/**
* Define native method prototypes
*/
public native int ngmsStartReceiver ();
public native int ngmsStopReceiver ();
public native int ngmsReceiveFrame (byte[] frameBytes);public native int ngmsStartSender (String remoteAddress);
public native int ngmsStopSender ();
public native int ngmsTransmitFrame (byte[] frameBytes);}
Once you have integrated the ExampleChat class into your own Android project ensure that all three of the native libraries exist in the project /libs directory so that they will be included into your.apk output.
libs/armeabi/libngms.so
libs/armeabi/libxcode.so
libs/armeabi/libngmsglue.so
This example produces video output to the destination IP address 10.0.0.1. To successfully enable two clients to stream video bi-directionally you will need to change this IP address to match your remote client.
Conclusion
The above example demonstrates how to stream live video from the camera object in real-time and how to receive a remote live stream and extract each decoded video frame for display. The same process flow can be applied to add audio streaming in-order to create a complete video chat application. â¨
To view the Nex Gen Media Server (NGMS) Embedded API Guide go to http://ngmsvid.com/develop.php
[ad_2]
Source by Hung B Nguyen
