Part 0: A high level intro to the latest AR framework
Before WWDC19, myself and other Augmented Reality (AR) enthusiasts were anticipating many updates to SceneKit to come along with some shiny new ARKit features; however this was not the case.
Instead Apple decided to create a whole new framework to implement AR, and dubbed this new framework RealityKit. Here we have a whole new rendering engine, made from the ground up with AR not just in mind, but as the top priority. Although there may be some features missing; with this as v1.0 I have high hopes that this will become the standard for high-performance AR. …
Voice and Video SDK v3.5.0 now available for on all supported native mobile and desktop platforms. See what’s new below!
Agora Native SDK 3.5.0 was released on 15 July 2021.
New Features ✨
Virtual Backgrounds 🧑🏫
Virtual backgrounds with Agora lets you select a colour or an image (PNG or JPG) as the background, behind a segmented person.
This feature has been available with the macOS and Windows SDKs since v3.4.5, and it is now available for Android and iOS in beta in v3.5.0.
The utility of this feature goes beyond just appearing to be on a sunny beach rather than in your home office. …
One of our aims is for you to integrate all Agora SDKs as easily as possible. Four years ago we first added CocoaPods support to Video Call SDK v1.10.0.
Now, we are happy to make integration even easier with Swift Package Manager support. This post shows you how to find and install the latest Swift packages from Agora in your app.
What is Swift Package Manager?
Swift Package Manager (SwiftPM) is a package dependency manager similar to CocoaPods and Carthage. It’s integrated with the Swift build system to automate the process of downloading, compiling, and linking dependencies. SwiftPM is included in Xcode, meaning there is…
When you’re building a real-time engagement application, a ton of metrics need to be monitored to deliver a smooth experience to the end user. There can be many challenges when debugging a suboptimal user experience: high CPU usage, low internet bandwidth, dropped frames, and so on. The in-call statistics can be used to monitor, maintain, and improve the user experience.
In this tutorial, we’ll add in-call stats to an iOS app, using the Agora UIKit CocoaPod to simplify the process. If you’re new to the Agora SDK, you can learn how to build a project from scratch by using the…
At last we can add a geometry or surface shader to an Entity in your RealityKit scene with the release of RealityKit 2! Coming with iOS 15/macOS 12.
What Is a Shader?
A shader is code that is passed to the renderer to then be run at specific stages in the rendering pipeline.
The two shaders we now have in RealityKit are geometry and surface shaders. Both of these are sampled in the sample code: Building an Immersive Experience with RealityKit.
At WWDC 2019 RealityKit was first released. It had some amazing features making Augmented Reality apps simple for all developers, with an Entity Component System (ECS). This was a big refresh from the native iOS framework SceneKit, which is what many developers were using to create their AR apps.
However a lot was missing from this brand new rendering system, and two years later many of those have been introduced at #WWDC21.
This guide will walk you through how to use one of these new features: procedural geometries.
What Are Procedural Geometries?
Most people understand procedural geometries as a way of writing an algorithm…
Augmented Reality highlights from WWDC 2021
Another WWDC announcement has come by, but this time RealityKit has received a noteworthy upgrade! Although in comparison, the ARKit upgrades are mostly performant.
All the topics mention in this article will be added to my ongoing RealityKit series over the coming months:
What Happened Last Year? (WWDC20)
As a reminder, last year the updates in ARKit 4.0 were Hand Pose, Depth API, Location Anchors, Face Tracking (for devices without FaceID). RealityKit 2.0 only came with Video Textures and a Debug Component for 3D models; as well as access to the ARKit updates. Here’s my summary from last year:
…
Agora’s main product is audio and video streaming services, which enable you to connect with friends, colleagues and teachers with super-low latency. But another Agora product enables you share messages between devices on the network: Agora RTM (real-time messaging) SDK. In this tutorial, you’ll learn some of the many things you can do with RTM.
This tutorial covers sending messages (such as text and photos) to others, rendering them with an open-source iOS framework called MessageKit, detecting who is in the RTM channel (with a feature to raise hands for users who need attention), and sharing files with the channel…
Voice and Video SDK v3.4.0 now available for on all major mobile and desktop platforms. See what’s new below!
Agora Native SDK 3.4.0 was released on April 16, 2021
New Features ✨
Virtual Metronome
Great for online music teaching or matching the beat with Karaoke. You can configure the digital metronome to the right pace for you.
Control Audio Effect Files
Control properties such as the effect position and start position of audio effect files. The previous method for playing audio effects has an extra parameter, startPos, which is required.
Degradation Preference
When you have limited bandwidth, Agora RTC Engine takes steps to maintain your connection to the call, including by either decreasing frame rate or number of pixels in the frames that come in. …
Agora UIKit Makes Video Streaming Easier Than Ever
When setting up your own streaming application with Agora, a few technical steps might slow you down. Now with Agora UIKit for iOS and macOS you can create an application as easily as placing a UIView, NSView or SwiftUI View.
