Hold on, good things ahead..

FNEF and Six Factor announce new Indigenous language app: secure, instantly available, and customizable

We have some great news to report about the work we’ve been doing over the past several months with Six Factor, our FNEF technology partner. Six Factor is western Canada’s Leading Google Cloud Partner, and together we are ready to field test the first generation of our FNEF Indigenous language app for smartphone, tablet, and desktop learning.

There are some key differences between our FNEF language app – which was built from the ground up to be secure and instantly available on a global level – and other language apps currently being used to curate and revitalize at-risk First Nations languages. For example, it has a “record and compare” feature that provides learners with a visual reference whereby the sound wave produced by their pronunciation of a word or words can be directly compared to the sound wave produced by a fluent speaker for enhanced pronunciation accuracy.

Likewise, content curation for our app is built around high quality video recording in tandem with audio recording – including individual words and alphabet pronunciations. Video offers a substantial improvement over merely recording high quality audio and adds a significant cultural benefit for future generations who will actually be able to see – as well as hear – Elders speaking the language they are learning (studies have shown that people learn faster, more effectively, and are more engaged when video is incorporated into learning).

In tech circles, our FNEF app is already turning heads with what it can do and how quickly it can do it (see the current technical specifications and cloud technology / security software used by our FNEF language app below at the bottom or skip directly to it here).

The technological sophistication of our FNEF app is based on our adoption of the significant, exponential advances in Google’s machine learning and “Deep Learning” technologies in recent years. Our goal is to fully harness this advanced learning power and bring it to bear on the challenge of revitalizing at-risk First Nation languages. Six Factor is continuing to work on ways to accomplish this for us while we field test this first generation of our app, which supplements the vitally important language nest, immersion, and mentoring work already underway in many First Nations communities.

Needless to say, security and instant global availability were high on our list of requirements for the FNEF app. Six Factor was able to achieve this for us by embracing Google’s Cloud Compute Engine technologies which allow for rapid scaling to meet instant demand without sacrificing performance or requiring the reworking of an app. The software architecture used to build our FNEF Language app is also framework based which means it is easily expandable to support multiple language sets.

So: What does this mean in practical terms? It means that the user interface for our desktop and mobile language app versions is easily customizable and personalized to the specific requirements of any First Nations language community; and new features such as learning environments based on virtual reality can be added in the future as various technologies advance.

There is an important bit of background to the language technology Six Factor is bringing to our app and to the goal of saving and revitalizing at-risk First Nations languages. The current state of Deep Learning (machine learning) technology makes it possible to “ingest” a thriving living language in just 30 days. Six Factor is looking to embed this same advanced machine learning technology into our FNEF app such that, with a sufficiently robust Indigenous language corpus, it will soon be possible for at-risk Indigenous languages to be machine-learned using our app simply by recording and absorbing the normal, everyday speech of fluent Elders. The implications for rapidly curating an at-risk First Nations language, and creating the conditions for a successful revitalization of that language, are profound.

In terms of the user-friendliness of our FNEF language app: Our app can access the microphone, camera, and photo library of a smartphone to record speech in HD. This allows a user to practice and participate in a game-like learning environment built into the app’s workflow experience. To protect end-user privacy, the practice sessions of each individual learner are stored in a randomly-named file folder structure. And before any public submissions – or uploaded recordings – are accepted into the application’s language dataset and made available to end users, they are stored in an encrypted staging folder for review and validation by a trainer or admin person. The use of each First Nation’s data is protected by a highly secure privacy policy whereby each user is granted role-based access and function permissions; e.g. user, trainer, Admin at the First Nation Government level.

For the tech folks – and without going too deeply into the technology – our FNEF language app is a cloud application written in PHP using the latest version of the Laravel framework. It serves HTML, Javascript and CSS to desktop users while delivering results to mobile users over RESTful API using a bearer or authorization token within the request. The mobile application of our FNEF app was developed in Angular, a powerful language that drives many of Google’s own services.

Application data is backed up nightly and stored in a secure Drive bucket for no less than thirty (30) days. Users can log in with a username and password, or by using Single sign on via their Google, Twitter, or Facebook accounts over oAuth.

All login and data upload requests require authorization, and CSRF tokens are used to protect against cross-server forgery attacks and cross-site scripting attacks. Using a rigid Content Security Policy, the HTML webpage serving the app’s front-end denies the loading of scripts from third party domains, and all libraries are stored locally. In other words, the data in our FNEF language app is heavily protected and very safe and secure.

Our FNEF language app and its environment are also penetration tested by a qualified cybersecurity firm on a regular schedule. The application inspects uploaded images/audio and validates content. Uploads are renamed using a randomized hash and stored in an isolated environment without the ability to execute code or access data on the server. The server and software are hardened to store the application in isolation, prohibiting access or visibility to non-essential memory or other sessions. The complete environment is resource/performance/process monitored and access/firewall logs are inspected by a cyber security specialist for suspicious activity with real time data streamed to a secure bucket.

As noted above, it is the significant advances that have been occurring almost monthly in Deep Learning and machine learning technologies in recent years that have made it possible to accomplish what we have accomplished with the first generation of our FNEF language app; and the rate at which these technologies are advancing is increasing exponentially every day. If we use the analogy of dog-years for the purpose of comparison (because everyone knows that seven dog-years equal one human-year), one human-year is currently the tech equivalent of roughly fifteen “application-years.” That’s how fast machine learning technology is changing and advancing.

There is no question that an urgent response is required to document and curate the language and knowledge held by fluent Elder speakers of Indigenous languages as rapidly as possible while they are still with us. Likewise, it’s no secret that this population grows smaller every day. Tapping into the exponential power of technology innovation is the key that will unlock our collective ability to meet this great challenge.

We want to thank Six Factor for partnering with us on this vitally important project and allowing us to achieve the key differences we were looking for with respect to the language apps currently being used to rescue at-risk First Nations languages. Six Factor shares our passion for saving these languages as well as our belief that an enormous technology boost is urgently needed – and right now – to preserve at-risk First Nations languages in a way that will allow future generations to successfully learn and revitalize these languages (and for all to learn from them and the unique knowledge, history, and wisdom they carry).

We are very proud of the language app we’ve created with Six Factor as our technology partner, but we’re not stopping there. Our ambition is to support First Nations communities and language champions in their language revitalization efforts by delivering the best in class technology innovation for language curation and revitalization on an ongoing basis.

Thank you again to Andy Parkins and everyone at Six Factor for joining us on this important journey. We are eternally grateful to all of you for what you have accomplished for us and for getting us closer to the goal of saving and revitalizing all at-risk First Nations languages.

FNEF

Current Technical Specifications and Cloud Technology / Security Software used by our FNEF language app:

  • Hardware: Scalable Google Compute Engine instances with Load Balancing
  • Operating System: Cloudlinux 7 with AtomicSecure Linux kernel/operating system hardening
  • Webserver Applications: Apache2 and PHP 7.2, with AtomicSecure Linux mod_security2 WAF rules enabled
  • Networking: Cloudflare (DDoS protection/performance), server inaccessible to internet directly, management processes (SSH/SFTP) available over the internet only via whitelisted IP and VPN, Google’s network firewall and a server side stateful inspection firewall
  • Malware Detection: Linux Malware Detect, Config Server’s CXS real-time malware scanner with mod_security2 hooks, rkhunter, chkrootkit and administrative inspection of processes

Six Factor is constantly evolving the Conceptual Technological Platform used to support our FNEF language application and they are fully embracing Google’s deep learning technologies

  • Six Factor is working to integrate Google’s TensorFlow NLP to provide real time translation to help students master the language through everyday examples
  • Six Factor is prototyping with ImmerseMe Virtual Reality to provide real engagement (see: https://immerseme.co/#explore and https://www.oculus.com/experiences/gear-vr/1272636489423125/ and https://www.oculus.com/experiences/gear-vr/1129567930394285/)
  • Six Factor is adding in the Google Speech API with NLP to create interactive learning sessions that learn the style of the student not just the living language structure to help them find the best and easiest way to blend their personal learning style with the teaching style of the app
  • Six Factor is also having a lot of fun building a Group Augmented Reality/VR/gamified experience to allow students or study groups to see the same translations in the environment around them and hear the words/letters spoken (broken into specific sounds & the sentence or word as a whole) to learn together – Identify objects or surfaces dynamically and translate them (see: https://experiments.withgoogle.com/ai/thing-translator and https://experiments.withgoogle.com/ai/giorgio-cam)
  • Six Factor is also big into making learning fun by bringing video game experiences into the app to deliver an engaging and interactive exploration of mythos, legends, and oral history (see: http://neveralonegame.com/). Six Factor’s goal is allow the student to explore by learning as a game in the First Nation language with English translations that allows the child and/or adult to learn the culture through an enjoyable game experience

 

FNEF

Please follow and like us: