Adding Secrets In Kaggle
In this article, we will walk you through the fantastic journey of learning Secrets in kaggle and along with that we also understanding the fundamentals of secrets in kaggle platform. Secrets are important concept in kaggle and we familiar with kaggle, experts and newbies are always preferred secrets for use.
Table of Content
- Adding and Using Secrets
- Using Google Cloud Services In Kaggle Notebooks
- Upgrading Your Kaggle Notebook to Google Cloud AI Notebooks
Adding and Using Secrets
Sometimes, you might need to add environment variables in your notebook and you would want to keep them secret, especially if you make the notebook public. Examples of such variables could be your connection token
for an experiment tracking service, like Neptune.ai or Weights & Biases or
various API secret keys or tokens. In this case, you would most probably
like to use one of the add-ons, Kaggle Secrets.
Upon selection of the Kaggle Secrets menu item, a window like the one in
the following screenshot will appear. In this pop-up window, you can add
new secrets by pressing the button Add a new secret. To include the secrets
with the current notebook, just check the check boxes near the secrets you
want to include.
In the preceding screenshot, three secrets two for a Twitter API connection
and one for Weights & Biases experiment tracking are selected. For each
selected secret, there is an additional generated line like in the Code Snippet
on the lower side of the window. You can copy all the generated lines to a
clipboard to include in your notebook code. After you press Done, you will
be able to paste the code into your notebook.
Once defined, the secret will be available to be included in any of your
notebooks. You can modify the text of one secret using the Edit button next
to its name. Note that when you fork one notebook that has secrets added,
the secrets won’t be associated anymore with the new notebook. To make
available the secrets to a new or forked notebook, it is enough to enter the
Secrets windows and press Done, in the context of editing that notebook. Of
course, if someone else is copying your notebook, that Kaggler Kaggle
user will have to set their own secrets. And if that Kaggler chooses to use
different names for the variables associated with the secrets, they will also
need to operate the change in the code. This feature allows you to not only
manage useful environment variables but also easily configure your
notebooks.
Using Google Cloud services in Kaggle Notebooks
To take advantage of Google Cloud services in your notebook, from the
Add-ons menu, select Google Cloud Services. In the dialog window that
opens, you can sync your Google account with your notebook by clicking on
Attach to Notebook. You can also select which Google Cloud services you
want to integrate with your Kaggle environment. Currently, Kaggle offers integration with Google Cloud Storage, BigQuery,
and AutoML. When using these services through Kaggle Notebooks, you
need to know that this will incur charges, according to the plan you have. If
you choose to use only public data with Big Query, you will not incur any
charges.
In the following figure, we show how you can select these services.
Select what Google Cloud services to use in Kaggle Notebooks. As
mentioned, you will need to link your Google Cloud account to Kaggle. In
the selection screen you can choose from BigQuery, Cloud Storage and
Google Cloud AI Platform Vertex AI Workbench. In our example, two
out of the three available services were selected.
Upgrading your Kaggle Notebook to Google
Cloud AI Notebooks
If you reach the limit of resources available for Kaggle Notebooks RAM,
the number of cores, or execution time, you can choose to promote your
notebook to Google Cloud AI Notebooks by exporting your notebook to
Google Cloud. Google Cloud AI Notebooks is a paid service from Google
Cloud, and it gives you access to computing resources in Google Cloud for
machine learning, using a notebook as an integrated development
environment (IDE). For this action, select File Upgrade to Google AI
Notebooks and you will be directed to the following window.
Follow this three step process, set up a billing enabled Google Cloud
project, set up your network instance and run your code. Your code can run without the resource limits now. Let’s see now how we can use a notebook to automatize the update of a
dataset.
Conclusion
Finally, we will learning the fundamentals of secrets in kaggle and whenever we spend time in learning any platform, than in few months we will master it. Kaggle is the best platform for data scientists and machine learning engineer and it doesn’t matter weather those folks are novice or experienced one.
