Just Ask: Introducing The Next Wave of Helpful AI Systems with Deus



Original Source Here

Just Ask: Introducing The Next Wave of Helpful AI Systems with Deus

Today we’re announcing our next-generation AI system, Deus, which can have high fidelity conversations to assist developers with a wide variety of coding tasks.

Software development is becoming ever more complex, but one thing is certain: it is always improving. At Describe.ai, we are moving towards smarter, more adaptive AI systems that can assist developers in a more natural and conversational way than ever before. That’s why, today, we’re giving a sneak preview of our next-generation AI model “Deus”, which can have high fidelity conversations to assist developers with code like never before. As we will detail later in this blog, we have much bigger aspirations than code assistance for the future generations of our models.

Before continuing, we would like to give new readers who are not familiar with the project a brief overview of our work on our original system, Gemini (link here, technical blog here). If you already are aware of our original work, please skip to the next section.

Gemini: Learning to do basic explanations of input

Gemini was an AI system for summarizing code into simple English sentences. Our higher-level goal with Gemini was straightforward: Can we teach a machine learning model to understand an input (like code) well enough that it can explain it in words? To us, being able to explain an input with a high degree of detail and accuracy is a critical part of true comprehension in machines; This concept could also be a critical part of being able to reason.

We decided to start with code explanations for two reasons:

  1. There is an abundance of code data that can be used
  2. Code is a simple testing sphere for our hypotheses. Since we understand computer programming (unlike other topics like medical imaging), we can verify confidently if we’re making progress or not.

The results of the original work were extremely promising. We were able to train a machine learning model on under 1 million code/explanation pairs to generate English explanations of code that were surprisingly high-quality. To give an example, here is an example of what a Gemini model was able to generate for a given code snippet:

# Gemini-1 exampleimport tensorflow as tf
from tensorflow.keras import datasets, layers, models
import matplotlib.pyplot as plt
(train_images, train_labels) = datasets.cifar10.load_data()
train_images = train_images / 255.0
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.summary()---------------------------------------------------------Generated output:This code is building a tensorflow model from the cifar 10 dataset.

In this example, you can see a simple code example for a relatively straightforward machine learning program in Python (if you aren’t technical, take our word for it!). Gemini was able to explain the code snippet it was provided in a concise one-sentence summary.

A great first step with some limitations

While we were incredibly impressed with this Gemini, there were some obvious limitations that were identified early on. Firstly, Gemini was not capable of explaining code snippets in high detail. For our example above, a concise one-sentence summary is acceptable, but for larger programs, Gemini would often “lose context” or miss major parts of the program. An example of this can be shown below:

# Gemini poor exampleimport json
import os
import stripe
stripe.api_key = 'api_key_123'from flask import Flask, render_template, jsonify, requestapp = Flask(__name__, static_folder='public',
static_url_path='', template_folder='public')
def calculate_order_amount(items):
return 1400
@app.route('/create-payment-intent', methods=['POST'])
def create_payment():
try:
data = json.loads(request.data)
intent = stripe.PaymentIntent.create(
amount=calculate_order_amount(data['items']),
currency='eur',
automatic_payment_methods={
'enabled': True,
},
)
return jsonify({
'clientSecret': intent['client_secret']
})
except Exception as e:
return jsonify(error=str(e)), 403


if __name__ == '__main__':
app.run(port=4242)
---------------------------------------------------------Generated output:This code is creating a payment server.

On a high level, Gemini is correct but misses major portions of relevant code that should be explained further. While there are many reasons why this might be, our main conclusion was that our training data consisted of examples that were fundamentally too simple or short.

Rethinking From the Ground Up: The Start of Deus

Gemini was a major step forward in terms of building a system that could understand inputs well enough to explain them in a sentence or more. However, this system was fundamentally limited in its ability to explain concepts in a longer format. We knew that if we wanted to improve Gemini, we had to rethink our approach from the ground up.

With our new system, Deus, we are introducing a strong text-based conversational interface instead of a traditional in/out model (like Gemini). What does this mean?

As they say, a picture is worth a thousand words. Let’s take a look at an example, comparing Gemini to Deus:

# Geminiimport tensorflow as tf
from tensorflow.keras import datasets, layers, models
import matplotlib.pyplot as plt
(train_images, train_labels) = datasets.cifar10.load_data()
train_images = train_images / 255.0
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.summary()---------------------------------------------------------Generated output:This code is building a tensorflow model from the cifar 10 dataset.

And now, Deus:

# Deusimport tensorflow as tf
from tensorflow.keras import datasets, layers, models
import matplotlib.pyplot as plt
(train_images, train_labels) = datasets.cifar10.load_data()
train_images = train_images / 255.0
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.summary()---------------------------------------------------------Conversation:>> User: How does this code work? I don't fully understand.Deus: Sure! On a high level, this code is building a convolutional neural net and returning a model object. The first layer in this network, `tf.keras.layers.Conv2D`, is a convolutional layer that is responsible for extracting features from the input image. It applies 32 filters (3x3 size) to the input image and returns an activation map that is 32x32x32 in shape. The next layer, `tf.keras.layers.MaxPooling2D`, is a pooling layer that acts as a sort of downsampling using max pooling. It looks at the 2x2 cells and picks the largest value from each of those cells, discarding the rest. Next, the model has a convolutional layer that applies 64 filters (3x3 size) to the activation map and returns an activation map that is 16x16x64 in shape. Then, the model has a pooling layer that reduces the size of the activation map by half. Finally, there is a convolutional layer that applies 64 filters (3x3 size) to the activation map and returns an activation map that is 8x8x64 in shape.>> User: Wow, that was a good explanation! But what is a convolutional neural net? I'm still a beginner.Deus: A convolutional neural network is a type of neural network that specializes in image recognition tasks.>> User: I got it, so CIFAR-10 is a image dataset?Deus: Right. CIFAR-10 is a dataset that consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images....

Simply no comparison.

With Deus, you can converse naturally about your code and ideas related to code. We believe this opens up a world of possibilities for high-level understanding systems, which we will detail later.

A shift in focus: learning to have conversations about inputs

In Gemini, we were focused on if a machine can explain what input is (in the first test case, code) in a sentence or more. While challenging, we now believe that this is a slightly flawed approach because of subjectivity. A singular explanation to one person may be sufficient, but to another person, it may not be in-depth enough; Quantifying metrics for this is equally challenging.

While quantifying conversation is not any easier, it does require a much higher level of understanding because it significantly increases the room for error. In other words, we believe that if something can have a conversation about something and get it correct more often than not, it shows a much higher comprehension.

Introducing Deus: End-to-end coding assistance

In this section, we want to break down our first intended use cases of Deus:

Explanation/summarization of code: Deus is capable of generating much higher-quality explanations of code. Similar to how Gemini worked, except now wrapped in an easy-to-use conversational interface.

import os
from flask import Flask, redirect, request
import stripe
stripe.api_key = 'apiKey-123'
app = Flask(__name__,
static_url_path='',
static_folder='public')
YOUR_DOMAIN = 'http://localhost:4242'@app.route('/create-checkout-session', methods=['POST'])
def create_checkout_session():
try:
checkout_session = stripe.checkout.Session.create(
line_items=[
{
'price': '',
'quantity': 1,
},
],
mode='payment',
success_url=YOUR_DOMAIN + '/success.html',
cancel_url=YOUR_DOMAIN + '/cancel.html',
)
except Exception as e:
return str(e)
return redirect(checkout_session.url, code=303)if __name__ == '__main__':
app.run(port=4242)
------------------------------------------------------>>> User: What is this code doing?Deus: On a high level, this code is creating a Flask app that uses the Stripe API to complete a payment session. The create_checkout_session takes the price that you have sent in, and creates a session with it. If for some reason the session could not be created, then it will return the error message. If the session does get created, then the user is redirected to a success page. This endpoint will be routed to when the Flask app is running using the PORT 4242.

Code question answering (in context and open domain): Developers can ask any question relating to a code snippet or about programming in general.

import os
from flask import Flask, redirect, request
import stripe
stripe.api_key = 'apiKey-123'
app = Flask(__name__,
static_url_path='',
static_folder='public')
YOUR_DOMAIN = 'http://localhost:4242'@app.route('/create-checkout-session', methods=['POST'])
def create_checkout_session():

# abbreviated
------------------------------------------------------>> User: Why do we start this on PORT 4242 instead of another PORT?Deus: Because 4242 is a port that is commonly used for web applications.>> User: I'm also confused, what is code 303?Deus: 303 is a redirect status code that sends a user to a different URL.

Code testing suggestions: Deus will suggest which tests to write based on your code. Developers write code and Deus will suggest unit tests and integration tests.

import os
from flask import Flask, redirect, request
import stripe
stripe.api_key = 'apiKey-123'
app = Flask(__name__,
static_url_path='',
static_folder='public')
YOUR_DOMAIN = 'http://localhost:4242'@app.route('/create-checkout-session', methods=['POST'])
def create_checkout_session():

# abbreviated
------------------------------------------------------>> User: Where might be some areas I should test this code?Deus: Some suggestions would be to test the GET and POST methods to make sure that you are getting a success message. You should also test to make sure that the stripe.checkout.Session object is created successfully.

Code bug fixing (beta): With Deus, we will offer a limited rollout of automatic bug fixing of code. While this will be separate from our main beta test pool, we are incredibly excited to offer this feature to a small group of testers.

Code concept explanation (beta): Deus is capable of explaining higher-level concepts to users. For example, if you ask “Why are Numpy arrays faster than regular lists”, you will get an explanation back from Deus.

And much more. We will continue to learn more about how the community uses Deus and how we can improve it, as the system is general purpose in nature.

Why we are doing this: Towards AI Systems That Can Truly Understand Language

Photo by JOHN TOWNER on Unsplash

The field of AI is constantly changing; Right now, this is particularly true for the field of natural language processing. Projects like GPT-3, PALM and many others are making rapid progress towards higher-level language understanding in machines. The baseline for evaluating AI systems is quickly becoming how general a single system is, instead of how well it can perform on a single task.

Today, we are also announcing our plans to work towards human-level language understanding with Deus. Why are we working towards this? The answer is simple: we believe that the next world-changing AI systems will be grounded within strong level of language understanding.

Our end goal with Deus is one system that can perform any language-related task at the level or better than a human. Some of these tasks can include conversation, translation, code synthesis, summarization and most importantly, accurate world knowledge. All of these tasks alone are difficult, but integrating them all into one learning algorithm is an extremely challenging task. However, if we are successful, it could be the biggest innovation in AI to date.

Interested in trying out Deus?

If you or your team is interested in using our Deus system for enhanced software development, please visit our platform at:

Where you can try out Deus with a $5 credit (limited time only).

Thank you for reading!

AI/ML

Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot

%d bloggers like this: