Sherloq Documentation

Hey all ✋

Here you will find all the information you need about Sherloq’s workings.

Do you have any suggestions for improving the product? Through Sherloq, send us feedback!

Are you looking for something else? Feel free to contact us at 📧hello@sherloqdata.com or via Slack.

Sherloq product tour

Profiling

Chrome Extension

Feedback & Contact

Login

Go to appsflyer.sherloqdata.com and sign in with your Gmail account.

Sherloq is now open to you!

If you have any login issues, please send an email to hello@sherloqdata.com or send us a message via Slack and we will assist you as fast as we can.

Search

Here you can search all over your tables located in S3 and all the queries that have been queried from AWS service since 1 January 2022.

In order to search on specific data assets (table or query, for meanwhile) choose on the toggle bar

Table View:

Here you can see the list of the tables that are located in S3.

This view contain the following details:

Table name
Who is the person who last queried
When the last queried was
In what database the table is located

🔍 For the meantime, the tables are searched only by their names.

🔍 This list is ordered by the amount of times this table has been queried.

🔍 In the top right corner there is a filter button that you can filter on the following criteria.

Tables Filters

You can combine many filters as you want. Pay attention that the filters are combined together.

🔍 For example, here are all the tables which contain “acc”, from a database which contains “biz” and was last queried by someone named “nadav”.

Here you can see the list of all the queries that have been queried from AWS service since 1 January 2022.

🔍 This view contain the following details:

The first of the query chars
Who is the person who last queried
When the last queried was
Tables that referenced by the query

🔍 In the meantime, the queries are searched by the query characters themselves as well as the name of the person who queried.

🔍 This queries list is ordered by the last queried time.

Query Filters

In the top right corner there is a filter button that you can filter on the following:

🔍 You can combine many filters as you want. Pay attention that the filters are combined together.

🔍 For example, here are all the queries that used the table “account” in the past 7 days, with dana as the user name who queried.

Profiling

When you click on a table/query from the search result, a table/query profiling will appear.
In a glance, the following information shows the usability of the tables/queries:

General info
Top users
Chat & Notes
Table and Fields Statistics
Most common fields
Most Joined Tables
Query usage

General Info

Database (DB) name
Date Last Queried from this table
Owner of the table – you can edit the owner of the table by clicking on it.
The user name who is the Last Queried from the table

Top Users

Top 5 name users and info
# uses – the amount of the user queried from this table
Last queried the user queried from – by clicking on this, you’ll see the query itself.

NOTE: You can filter the top users by department/business unit on the top-right list on the corner.

Chat & Notes

Through the chat box, you can note and comment on the table and share your knowledge and questions.

Table and Fields Statistics

all the fields (and these type) in table order by the amount of uses. In order to see all the fields, you can click on “View all x fields”

Note: the number % refers by this calculation –

{The number of queries that reference this field / the number of queries that reference this table}

Most common fields

You can see the top three fields that are used by every operator (Select, Where, etc).

In the brackets, you can see the number of the field’s used by this operator.

Most Joined Tables

You can see what tables (by JOIN operators) are connected to what based on the platform that your organization uses.

Query usage

You can see two sections of queries that refer to this table here:

Most Used Query
Last Query Run

There is no automation queries process here, only code written by people.

By clicking the “View All” button, you can see all queries that referenced this table.

Lists

Within the list menu, you can easily access your saved queries and tables.

It allows you to manage your own folder and save what you regularly use.

🔍 By searching, you can also see the list of your friends that you can take inspiration from, as well as access data assets more quickly.

🔍You can add/remove the asset from your list (don’t worry! Removing won’t delete the asset, only from your list) by entering the table/query profiling and clicking “Add To My List” / “Remove From My List”.

Chrome extension

You can download the Sherloq chrome extension here:

After adding to chrome press OK when it will ask you to modify the athena view.

To begin using the Sherloq Chrome extension, click the Sherloq opener on the Athena AWS site, then login into your company’s Gmail account.

The Sherloq opener queries list can be found on the right side of Athena AWS once you log in. Also, you’ll see our new save button.

🔍 You can save your query by marketing the SQL code and then click on the Sherloq-save button. You can also add a name to this query and a description in order to make it clearer for the next time you use it and be able to search for it.

🔍 In order to load all your queries, press on the Sherloq-button on the right side and choose your query by clicking on.

*note: the default query title will be the name or the description of the query. If there’s no name/description it will be by the first characters of the query.