Lets explore a movie

​

Setup

In this tutorial, we will go through an example of how to index a movie and search through it. We will explore searching via text, images and video. We will also explore how to use the answer engine to generate answers to our questions based on the contents of the movie.

To keep things simple, this example uses a single video in the index. However, the power of AskVideos scales to any number of videos in the index. That is, you can search through and get answers from multiple videos at once.

Let’s get right into it. For this example, I’ll be using the movie No Country For Old Men as an example.

​

Indexing the movie

The first step is to create an index to store and process the movie.

index_name = 'movie_example'

# Create an index
results = client.create_index(index_name)
print(results)
'''
{'index_name': 'movie_example', 'success': True}
'''

# List all available indexes
results = client.list_indices()
print(results)
'''
{'indices': ["movie_example"]}
'''

Cool, we should now have an empty index that we can add the movie to.

# Index a video
movie_file = 'no_country_for_old_men.mp4'

results = client.index(
            index_name,
            movie_file)

print(results)
'''
'''

If you already have a transcript .vtt file, we can also add it while indexing. You may also add additional metadata.

# Index a video
movie_file = 'no_country_for_old_men.mp4'
transcript_path = 'no_country_for_old_men.vtt'
metadata = {
  'title': "No Country For Old Men",
  'description': "A classic thriller set in the deserts of Texas",
  'info': {}
}

results = client.index(
            index_name,
            movie_file,
            transcript_path=transcript_path)

print(results)
'''
'''

If you navigate to your dashboard at app.askvideos.com, you should now see that the index has been created. If you click on the index name, you should also see that the video has been created.

​

Finding scenes

​

Using text

One of the cool things you can do with AskVideos is search through the visual content of the video with text.

For example if we wanted to find all the scenes where someone’s wearing a cowboy hat (it is set in Texas after all!), we could enter that as the query to the search API.

# Find scenes with cowboy hats
search_term = 'cowboy hats'

results = client.search(
            index_name,
            search_term,
            top_k=10)
print(results)
'''
'''

We should see that it has returned 10 results since we specified top_k to be 10.

Each result has an associated confidence score along with start and end timestamps.

Here are what the results look like:

​

Using images

Now let’s try using an image to search!

Images can be powerful search queries when we are trying to describe a vibe or particular scene with a lot of things happening in it.

Let’s try to find a specific scene that is shown in the trailer using an image.

# Find scenes with an image
image_path = 'img.jpg'

results = client.search_by_image(
            index_name,
            image_path=image_path,
            top_k=10)
print(results)
'''
'''

As before, the result is a list of highest scoring sections of the movie.

This scene occurs at 22:30 of the movie - case closed!

​

Using videos

If we have a video we would like to use as our search query, we can do that too!

# Find scenes with an video
video_path = 'video.mp4'

results = client.search_by_video(
            index_name,
            video_path=video_path,
            start_seconds=10,
            end_seconds=14,
            top_k=10)
print(results)
'''
'''

Here, we have two addition variables start_seconds and end_seconds that specify which segment of the query video to use for the search. Note that these variables relate to the video we are using as our query and not the other videos that are in the index.

The results of the search again show the scene we were looking for!

​

Answer Engine

We can also use language models in the AskVideos API to do various tasks ask questions and synthesize answers from our videos or ‘chat’ with a video.

query = '''
Which city does Llewelyn tell Carla Jean to go to?
'''

results = client.answer(
            index_name,
            query)

Here’s an example where the API answers a question about the movie and provides a precise timestamp to where the information is contained.

​

Next steps

Check out our other examples or dive deeper into the api