{"id":53555,"date":"2025-07-29T15:16:37","date_gmt":"2025-07-29T05:16:37","guid":{"rendered":"https:\/\/www.cloudproinc.com.au\/?p=53555"},"modified":"2025-07-29T15:16:39","modified_gmt":"2025-07-29T05:16:39","slug":"counting-tokens-using-the-openai-python-sdk","status":"publish","type":"post","link":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/","title":{"rendered":"Counting Tokens Using the OpenAI Python SDK"},"content":{"rendered":"\n<p>This post provides a comprehensive guide on counting tokens using the OpenAI Python SDK, covering Python virtual environments, managing your OpenAI API key securely, and the role of the <code>requirements.txt<\/code> file.<\/p>\n\n\n\n<!--more-->\n\n\n\n<p>In the world of Large Language Models (LLMs) and Artificial Intelligence (AI), the term &#8220;token&#8221; frequently arises. Tokens are units of text used to measure the length of the inputs and outputs in LLM APIs such as OpenAI&#8217;s GPT models. Understanding how to count tokens accurately is crucial for effective API use, budgeting, and maintaining efficiency. <\/p>\n\n\n\n<div class=\"wp-block-yoast-seo-table-of-contents yoast-table-of-contents\"><h2>Table of contents<\/h2><ul><li><a href=\"#h-why-counting-tokens-matters\" data-level=\"2\">Why Counting Tokens Matters<\/a><\/li><li><a href=\"#h-setting-up-your-python-virtual-environment\" data-level=\"2\">Setting Up Your Python Virtual Environment<\/a><ul><li><a href=\"#h-step-1-create-a-virtual-environment\" data-level=\"3\">Step 1: Create a Virtual Environment<\/a><\/li><li><a href=\"#h-step-2-activate-the-virtual-environment\" data-level=\"3\">Step 2: Activate the Virtual Environment<\/a><\/li><\/ul><\/li><li><a href=\"#h-using-a-env-file-for-your-api-key\" data-level=\"2\">Using a .env File for Your API Key<\/a><ul><li><a href=\"#h-step-1-create-the-env-file\" data-level=\"3\">Step 1: Create the .env File<\/a><\/li><li><a href=\"#h-step-2-load-the-api-key-into-your-application\" data-level=\"3\">Step 2: Load the API Key into Your Application<\/a><\/li><\/ul><\/li><li><a href=\"#h-installing-the-openai-sdk-with-requirements-txt\" data-level=\"2\">Installing the OpenAI SDK with requirements.txt<\/a><ul><li><a href=\"#h-step-1-create-requirements-txt\" data-level=\"3\">Step 1: Create requirements.txt<\/a><\/li><li><a href=\"#h-step-2-install-packages\" data-level=\"3\">Step 2: Install Packages<\/a><\/li><\/ul><\/li><li><a href=\"#h-counting-tokens-with-openai-sdk-and-tiktoken\" data-level=\"2\">Counting Tokens with OpenAI SDK and TikToken<\/a><\/li><li><a href=\"#h-advanced-token-management\" data-level=\"2\">Advanced Token Management<\/a><\/li><li><a href=\"#h-best-practices\" data-level=\"2\">Best Practices<\/a><\/li><li><a href=\"#h-conclusion\" data-level=\"2\">Conclusion<\/a><\/li><\/ul><\/div>\n\n\n\n<h2 class=\"wp-block-heading has-black-color has-text-color has-link-color wp-elements-c6d1e5ac3048c25e964c677b534d8b68\" id=\"h-why-counting-tokens-matters\">Why Counting Tokens Matters<\/h2>\n\n\n\n<p>Tokens are essential because they directly affect how models process and bill your requests. OpenAI APIs have limits based on tokens rather than characters or words. Counting tokens helps you:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Estimate costs effectively.<\/li>\n\n\n\n<li>Avoid exceeding model context limits, which could cause API errors.<\/li>\n\n\n\n<li>Optimize your prompts and responses for better performance and reduced costs.<\/li>\n<\/ul>\n\n\n\n<p>Tokens usually correspond to words or subwords. For example, &#8220;ChatGPT&#8221; might count as two tokens: &#8220;Chat&#8221; and &#8220;GPT&#8221;. Hence, token counting is essential for managing interactions with the API efficiently.<\/p>\n\n\n\n<h2 class=\"wp-block-heading has-black-color has-text-color has-link-color wp-elements-fa0913fb32da64a1dff91031dfd26db9\" id=\"h-setting-up-your-python-virtual-environment\">Setting Up Your Python Virtual Environment<\/h2>\n\n\n\n<p>A Python virtual environment allows you to isolate dependencies required for different projects. This prevents conflicts between packages.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-black-color has-text-color has-link-color wp-elements-4d748c534e958448dab0f217999b0e49\" id=\"h-step-1-create-a-virtual-environment\">Step 1: Create a Virtual Environment<\/h3>\n\n\n\n<p>To create a virtual environment, open your terminal and run:<\/p>\n\n\n\n<pre class=\"wp-block-code has-white-color has-black-background-color has-text-color has-background has-link-color wp-elements-96f018da85cdacae1fa2abb1816fd0ad\"><code>python -m venv openai-token-counter-env<\/code><\/pre>\n\n\n\n<p>This creates a new directory named <code>openai-token-counter-env<\/code> containing isolated Python binaries and libraries.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-black-color has-text-color has-link-color wp-elements-80f61bce1a2e94ee132ffcd1363a98c7\" id=\"h-step-2-activate-the-virtual-environment\">Step 2: Activate the Virtual Environment<\/h3>\n\n\n\n<p>Activate the virtual environment to ensure you&#8217;re using the correct Python interpreter and libraries:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"has-black-color has-text-color has-link-color wp-elements-ec2c8f558a84b1bcf5b79b12c8eef522\"><strong>Windows:<\/strong><\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code has-white-color has-black-background-color has-text-color has-background has-link-color wp-elements-f67d20e15afd9542610c18810823fcf9\"><code>openai-token-counter-env\\Scripts\\activate<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"has-black-color has-text-color has-link-color wp-elements-ab5a25963ee7871ba709be62b34a957a\"><strong>macOS\/Linux:<\/strong><\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code has-white-color has-black-background-color has-text-color has-background has-link-color wp-elements-0fea7a1ad28ef9e2438fcd898ccc9c7e\"><code>source openai-token-counter-env\/bin\/activate<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading has-black-color has-text-color has-link-color wp-elements-bf5699a8ab0301426edb2d58b7bcd8f9\" id=\"h-using-a-env-file-for-your-api-key\">Using a <code>.env<\/code> File for Your API Key<\/h2>\n\n\n\n<p>Storing sensitive information like API keys securely is crucial. Using a <code>.env<\/code> file is a safe practice to manage these keys and other configurations.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-black-color has-text-color has-link-color wp-elements-b74d6b692a7ede236e74ab77a3112c2f\" id=\"h-step-1-create-the-env-file\">Step 1: Create the <code>.env<\/code> File<\/h3>\n\n\n\n<p>Inside your project directory, create a new file named <code>.env<\/code> and add your OpenAI API key:<\/p>\n\n\n\n<pre class=\"wp-block-code has-white-color has-black-background-color has-text-color has-background has-link-color wp-elements-fdf9030fb5e3a97f08887ef45a09bfc5\"><code>OPENAI_API_KEY=your_openai_api_key_here<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading has-black-color has-text-color has-link-color wp-elements-52fc314c4770bd66c08bdb8339c7426d\" id=\"h-step-2-load-the-api-key-into-your-application\">Step 2: Load the API Key into Your Application<\/h3>\n\n\n\n<p>Install the <code>python-dotenv<\/code> package to load environment variables from your <code>.env<\/code> file:<\/p>\n\n\n\n<pre class=\"wp-block-code has-white-color has-black-background-color has-text-color has-background has-link-color wp-elements-66e0e9210c6fb1def49e4f669955859a\"><code>pip install python-dotenv<\/code><\/pre>\n\n\n\n<p>Then, load your API key in your Python script:<\/p>\n\n\n\n<pre class=\"wp-block-code has-white-color has-black-background-color has-text-color has-background has-link-color wp-elements-85e13cb0fbdaea5bb55e0abeba108400\"><code>from dotenv import load_dotenv\nimport os\n\nload_dotenv()\nopenai_api_key = os.getenv(\"OPENAI_API_KEY\")\n<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading has-black-color has-text-color has-link-color wp-elements-dbd828eb0ac68103d59cc6245decae99\" id=\"h-installing-the-openai-sdk-with-requirements-txt\">Installing the OpenAI SDK with <code>requirements.txt<\/code><\/h2>\n\n\n\n<p>Using a <code>requirements.txt<\/code> file simplifies dependency management, enabling quick setup for any user.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-black-color has-text-color has-link-color wp-elements-8a261e338f82e54df948d130a7a7fc97\" id=\"h-step-1-create-requirements-txt\">Step 1: Create <code>requirements.txt<\/code><\/h3>\n\n\n\n<p>In your project directory, create a file named <code>requirements.txt<\/code> and list the required packages:<\/p>\n\n\n\n<pre class=\"wp-block-code has-white-color has-black-background-color has-text-color has-background has-link-color wp-elements-cbdd44f9a2c151dc872e650b341ff9f0\"><code>openai\npython-dotenv\ntiktoken<\/code><\/pre>\n\n\n\n<p>Here:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>openai<\/code> is the official OpenAI Python SDK.<\/li>\n\n\n\n<li><code>python-dotenv<\/code> is for loading environment variables.<\/li>\n\n\n\n<li><code>tiktoken<\/code> is OpenAI\u2019s library for accurately counting tokens.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading has-black-color has-text-color has-link-color wp-elements-0aa03fab6b940ef11fff6f7a19e968d5\" id=\"h-step-2-install-packages\">Step 2: Install Packages<\/h3>\n\n\n\n<p>Run the following command to install packages from your <code>requirements.txt<\/code> file:<\/p>\n\n\n\n<pre class=\"wp-block-code has-white-color has-black-background-color has-text-color has-background has-link-color wp-elements-5c2d543dd533e960d6aad7ccb163bf49\"><code>pip install -r requirements.txt<\/code><\/pre>\n\n\n\n<p>This command ensures all necessary packages are installed quickly and efficiently.<\/p>\n\n\n\n<h2 class=\"wp-block-heading has-black-color has-text-color has-link-color wp-elements-4b4d2f2772edde980aa4d995dce1aac2\" id=\"h-counting-tokens-with-openai-sdk-and-tiktoken\">Counting Tokens with OpenAI SDK and TikToken<\/h2>\n\n\n\n<p>Here&#8217;s a simple example of how to use TikToken to count tokens for text input:<\/p>\n\n\n\n<pre class=\"wp-block-code has-white-color has-black-background-color has-text-color has-background has-link-color wp-elements-8013b840b1a4405df05e836cef76576e\"><code>import openai\nimport tiktoken\nfrom dotenv import load_dotenv\nimport os\n\nload_dotenv()\nopenai.api_key = os.getenv(\"OPENAI_API_KEY\")\n\n# Initialize encoding for the desired model\nencoding = tiktoken.encoding_for_model(\"gpt-4\")\n\n# Define your text\ntext = \"Hello, this is an example text to count tokens.\"\n\n# Count tokens\ntoken_count = len(encoding.encode(text))\n\nprint(f\"Number of tokens: {token_count}\")\n<\/code><\/pre>\n\n\n\n<p>In this snippet:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>We load the OpenAI API key securely.<\/li>\n\n\n\n<li>TikToken is used for encoding text specific to the chosen GPT model.<\/li>\n\n\n\n<li>The text is encoded into tokens, and the length of this encoding gives us the token count.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading has-black-color has-text-color has-link-color wp-elements-335d2d88d4347996a0a534cbbff6a7ae\" id=\"h-advanced-token-management\">Advanced Token Management<\/h2>\n\n\n\n<p>Beyond counting tokens, you can monitor token usage dynamically within your API interactions:<\/p>\n\n\n\n<pre class=\"wp-block-code has-white-color has-black-background-color has-text-color has-background has-link-color wp-elements-1fba03d5ff2e79e5ab81f99a14c60dd2\"><code>response = openai.ChatCompletion.create(\n    model=\"gpt-4\",\n    messages=&#91;{\"role\": \"user\", \"content\": text}]\n)\n\nusage = response.usage\nprint(f\"Prompt tokens: {usage.prompt_tokens}\")\nprint(f\"Completion tokens: {usage.completion_tokens}\")\nprint(f\"Total tokens: {usage.total_tokens}\")\n<\/code><\/pre>\n\n\n\n<p>This helps track tokens spent on each request, aiding cost management and optimization.<\/p>\n\n\n\n<h2 class=\"wp-block-heading has-black-color has-text-color has-link-color wp-elements-37e2041d55498b5c6f4fb319ebef2d8f\" id=\"h-best-practices\">Best Practices<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Always store API keys securely using <code>.env<\/code> files.<\/li>\n\n\n\n<li>Regularly update your <code>requirements.txt<\/code> to maintain dependencies.<\/li>\n\n\n\n<li>Use virtual environments to isolate and manage your project-specific packages efficiently.<\/li>\n\n\n\n<li>Check token counts proactively to avoid API rate limits and manage expenses effectively.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading has-black-color has-text-color has-link-color wp-elements-4c467fb51ff3609a0580c973dcea5aef\" id=\"h-conclusion\">Conclusion<\/h2>\n\n\n\n<p>Counting tokens using the OpenAI Python SDK and TikToken is straightforward yet crucial for managing API interactions efficiently. Using Python virtual environments, <code>.env<\/code> files, and proper package management with <code>requirements.txt<\/code>, you ensure your development environment is robust, secure, and easily maintainable. Adopting these best practices helps maintain smooth, cost-effective interactions with OpenAI\u2019s powerful AI models.<\/p>\n\n\n\n<ul class=\"wp-block-yoast-seo-related-links yoast-seo-related-links\">\n<li><a href=\"https:\/\/cloudproinc.com.au\/index.php\/2024\/10\/14\/auto-generate-azure-bearer-token-with-postman\/\">Auto-Generate Azure Bearer Token with Postman<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.cloudproinc.com.au\/index.php\/2024\/10\/11\/create-an-app-registration-for-microsoft-azure-sdk-for-net\/\">Create an App Registration for Microsoft Azure SDK for .NET<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/cloudproinc.com.au\/index.php\/2025\/07\/09\/top-vs-code-extensions-for-developers-and-devops-engineers\/\">Top VS Code Extensions for Developers and DevOps Engineers<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.cloudproinc.com.au\/index.php\/2024\/10\/07\/generate-dall-e-images-with-net-c-console-application\/\">Generate DALL-E Images with .NET C# Console Application<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.cloudproinc.com.au\/index.php\/2024\/07\/25\/deploy-azure-openai-gpt-4o-resource-and-model-using-bicep\/\">Deploy Azure OpenAI GPT-4o Resource and Model using Bicep<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>This post provides a comprehensive guide on counting tokens using the OpenAI Python SDK, covering Python virtual environments, managing your OpenAI API key securely, and the role of the requirements.txt file.<\/p>\n","protected":false},"author":1,"featured_media":53556,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_yoast_wpseo_focuskw":"Counting Tokens Using the OpenAI Python SDK","_yoast_wpseo_title":"","_yoast_wpseo_metadesc":"Learn how to count tokens using the OpenAI Python SDK and manage your API key securely for effective API use.","_yoast_wpseo_opengraph-title":"","_yoast_wpseo_opengraph-description":"","_yoast_wpseo_twitter-title":"","_yoast_wpseo_twitter-description":"","_et_pb_use_builder":"off","_et_pb_old_content":"","_et_gb_content_width":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[24,13,53],"tags":[],"class_list":["post-53555","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","category-blog","category-openai"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.3 (Yoast SEO v27.4) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Counting Tokens Using the OpenAI Python SDK - CPI Consulting<\/title>\n<meta name=\"description\" content=\"Learn how to count tokens using the OpenAI Python SDK and manage your API key securely for effective API use.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Counting Tokens Using the OpenAI Python SDK\" \/>\n<meta property=\"og:description\" content=\"Learn how to count tokens using the OpenAI Python SDK and manage your API key securely for effective API use.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/\" \/>\n<meta property=\"og:site_name\" content=\"CPI Consulting\" \/>\n<meta property=\"article:published_time\" content=\"2025-07-29T05:16:37+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-07-29T05:16:39+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/cloudproinc.azurewebsites.net\/wp-content\/uploads\/2025\/07\/image-23.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"768\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"CPI Staff\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"CPI Staff\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/index.php\\\/2025\\\/07\\\/29\\\/counting-tokens-using-the-openai-python-sdk\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/index.php\\\/2025\\\/07\\\/29\\\/counting-tokens-using-the-openai-python-sdk\\\/\"},\"author\":{\"name\":\"CPI Staff\",\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/#\\\/schema\\\/person\\\/192eeeb0ce91062126ce3822ae88fe6e\"},\"headline\":\"Counting Tokens Using the OpenAI Python SDK\",\"datePublished\":\"2025-07-29T05:16:37+00:00\",\"dateModified\":\"2025-07-29T05:16:39+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/index.php\\\/2025\\\/07\\\/29\\\/counting-tokens-using-the-openai-python-sdk\\\/\"},\"wordCount\":707,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/index.php\\\/2025\\\/07\\\/29\\\/counting-tokens-using-the-openai-python-sdk\\\/#primaryimage\"},\"thumbnailUrl\":\"\\\/wp-content\\\/uploads\\\/2025\\\/07\\\/image-23.png\",\"articleSection\":[\"AI\",\"Blog\",\"OpenAI\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/index.php\\\/2025\\\/07\\\/29\\\/counting-tokens-using-the-openai-python-sdk\\\/#respond\"]}],\"accessibilityFeature\":[\"tableOfContents\"]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/index.php\\\/2025\\\/07\\\/29\\\/counting-tokens-using-the-openai-python-sdk\\\/\",\"url\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/index.php\\\/2025\\\/07\\\/29\\\/counting-tokens-using-the-openai-python-sdk\\\/\",\"name\":\"Counting Tokens Using the OpenAI Python SDK - CPI Consulting\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/index.php\\\/2025\\\/07\\\/29\\\/counting-tokens-using-the-openai-python-sdk\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/index.php\\\/2025\\\/07\\\/29\\\/counting-tokens-using-the-openai-python-sdk\\\/#primaryimage\"},\"thumbnailUrl\":\"\\\/wp-content\\\/uploads\\\/2025\\\/07\\\/image-23.png\",\"datePublished\":\"2025-07-29T05:16:37+00:00\",\"dateModified\":\"2025-07-29T05:16:39+00:00\",\"description\":\"Learn how to count tokens using the OpenAI Python SDK and manage your API key securely for effective API use.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/index.php\\\/2025\\\/07\\\/29\\\/counting-tokens-using-the-openai-python-sdk\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/index.php\\\/2025\\\/07\\\/29\\\/counting-tokens-using-the-openai-python-sdk\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/index.php\\\/2025\\\/07\\\/29\\\/counting-tokens-using-the-openai-python-sdk\\\/#primaryimage\",\"url\":\"\\\/wp-content\\\/uploads\\\/2025\\\/07\\\/image-23.png\",\"contentUrl\":\"\\\/wp-content\\\/uploads\\\/2025\\\/07\\\/image-23.png\",\"width\":1024,\"height\":768},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/index.php\\\/2025\\\/07\\\/29\\\/counting-tokens-using-the-openai-python-sdk\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/cloudproinc.com.au\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Counting Tokens Using the OpenAI Python SDK\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/#website\",\"url\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/\",\"name\":\"Cloud Pro Inc - CPI Consulting Pty Ltd\",\"description\":\"Cloud, AI &amp; Cybersecurity Consulting | Melbourne\",\"publisher\":{\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/#organization\",\"name\":\"Cloud Pro Inc - Cloud Pro Inc - CPI Consulting Pty Ltd\",\"url\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"\\\/wp-content\\\/uploads\\\/2022\\\/01\\\/favfinalfile.png\",\"contentUrl\":\"\\\/wp-content\\\/uploads\\\/2022\\\/01\\\/favfinalfile.png\",\"width\":500,\"height\":500,\"caption\":\"Cloud Pro Inc - Cloud Pro Inc - CPI Consulting Pty Ltd\"},\"image\":{\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/#\\\/schema\\\/person\\\/192eeeb0ce91062126ce3822ae88fe6e\",\"name\":\"CPI Staff\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/2d96eeb53b791d92c8c50dd667e3beec92c93253bb6ff21c02cfa8ca73665c70?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/2d96eeb53b791d92c8c50dd667e3beec92c93253bb6ff21c02cfa8ca73665c70?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/2d96eeb53b791d92c8c50dd667e3beec92c93253bb6ff21c02cfa8ca73665c70?s=96&d=mm&r=g\",\"caption\":\"CPI Staff\"},\"sameAs\":[\"http:\\\/\\\/www.cloudproinc.com.au\"],\"url\":\"https:\\\/\\\/cloudproinc.azurewebsites.net\\\/index.php\\\/author\\\/cpiadmin\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Counting Tokens Using the OpenAI Python SDK - CPI Consulting","description":"Learn how to count tokens using the OpenAI Python SDK and manage your API key securely for effective API use.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/","og_locale":"en_US","og_type":"article","og_title":"Counting Tokens Using the OpenAI Python SDK","og_description":"Learn how to count tokens using the OpenAI Python SDK and manage your API key securely for effective API use.","og_url":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/","og_site_name":"CPI Consulting","article_published_time":"2025-07-29T05:16:37+00:00","article_modified_time":"2025-07-29T05:16:39+00:00","og_image":[{"width":1024,"height":768,"url":"https:\/\/cloudproinc.azurewebsites.net\/wp-content\/uploads\/2025\/07\/image-23.png","type":"image\/png"}],"author":"CPI Staff","twitter_card":"summary_large_image","twitter_misc":{"Written by":"CPI Staff","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/#article","isPartOf":{"@id":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/"},"author":{"name":"CPI Staff","@id":"https:\/\/cloudproinc.azurewebsites.net\/#\/schema\/person\/192eeeb0ce91062126ce3822ae88fe6e"},"headline":"Counting Tokens Using the OpenAI Python SDK","datePublished":"2025-07-29T05:16:37+00:00","dateModified":"2025-07-29T05:16:39+00:00","mainEntityOfPage":{"@id":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/"},"wordCount":707,"commentCount":0,"publisher":{"@id":"https:\/\/cloudproinc.azurewebsites.net\/#organization"},"image":{"@id":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/#primaryimage"},"thumbnailUrl":"\/wp-content\/uploads\/2025\/07\/image-23.png","articleSection":["AI","Blog","OpenAI"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/#respond"]}],"accessibilityFeature":["tableOfContents"]},{"@type":"WebPage","@id":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/","url":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/","name":"Counting Tokens Using the OpenAI Python SDK - CPI Consulting","isPartOf":{"@id":"https:\/\/cloudproinc.azurewebsites.net\/#website"},"primaryImageOfPage":{"@id":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/#primaryimage"},"image":{"@id":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/#primaryimage"},"thumbnailUrl":"\/wp-content\/uploads\/2025\/07\/image-23.png","datePublished":"2025-07-29T05:16:37+00:00","dateModified":"2025-07-29T05:16:39+00:00","description":"Learn how to count tokens using the OpenAI Python SDK and manage your API key securely for effective API use.","breadcrumb":{"@id":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/#primaryimage","url":"\/wp-content\/uploads\/2025\/07\/image-23.png","contentUrl":"\/wp-content\/uploads\/2025\/07\/image-23.png","width":1024,"height":768},{"@type":"BreadcrumbList","@id":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/07\/29\/counting-tokens-using-the-openai-python-sdk\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/cloudproinc.com.au\/"},{"@type":"ListItem","position":2,"name":"Counting Tokens Using the OpenAI Python SDK"}]},{"@type":"WebSite","@id":"https:\/\/cloudproinc.azurewebsites.net\/#website","url":"https:\/\/cloudproinc.azurewebsites.net\/","name":"Cloud Pro Inc - CPI Consulting Pty Ltd","description":"Cloud, AI &amp; Cybersecurity Consulting | Melbourne","publisher":{"@id":"https:\/\/cloudproinc.azurewebsites.net\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/cloudproinc.azurewebsites.net\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/cloudproinc.azurewebsites.net\/#organization","name":"Cloud Pro Inc - Cloud Pro Inc - CPI Consulting Pty Ltd","url":"https:\/\/cloudproinc.azurewebsites.net\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/cloudproinc.azurewebsites.net\/#\/schema\/logo\/image\/","url":"\/wp-content\/uploads\/2022\/01\/favfinalfile.png","contentUrl":"\/wp-content\/uploads\/2022\/01\/favfinalfile.png","width":500,"height":500,"caption":"Cloud Pro Inc - Cloud Pro Inc - CPI Consulting Pty Ltd"},"image":{"@id":"https:\/\/cloudproinc.azurewebsites.net\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/cloudproinc.azurewebsites.net\/#\/schema\/person\/192eeeb0ce91062126ce3822ae88fe6e","name":"CPI Staff","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/2d96eeb53b791d92c8c50dd667e3beec92c93253bb6ff21c02cfa8ca73665c70?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/2d96eeb53b791d92c8c50dd667e3beec92c93253bb6ff21c02cfa8ca73665c70?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/2d96eeb53b791d92c8c50dd667e3beec92c93253bb6ff21c02cfa8ca73665c70?s=96&d=mm&r=g","caption":"CPI Staff"},"sameAs":["http:\/\/www.cloudproinc.com.au"],"url":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/author\/cpiadmin\/"}]}},"jetpack_featured_media_url":"\/wp-content\/uploads\/2025\/07\/image-23.png","jetpack-related-posts":[{"id":53705,"url":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/08\/24\/how-to-use-the-tiktoken-tokenizer\/","url_meta":{"origin":53555,"position":0},"title":"How to Use the tiktoken Tokenizer","author":"CPI Staff","date":"August 24, 2025","format":false,"excerpt":"In this article, we\u2019ll explore How to Use the tiktoken Tokenizer, why it matters, and practical ways you can apply it in your projects to better control prompts, estimate API costs, and optimize large text inputs. Table of contentsWhy Tokenization MattersInstalling tiktokenBasic UsageDecoding Tokens Back to TextCounting TokensUsing Model-Specific EncodingsWorking\u2026","rel":"","context":"In &quot;AI&quot;","block_context":{"text":"AI","link":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/category\/ai\/"},"img":{"alt_text":"","src":"\/wp-content\/uploads\/2025\/08\/how-to-use-the-tiktoken-tokenizer.png","width":350,"height":200,"srcset":"\/wp-content\/uploads\/2025\/08\/how-to-use-the-tiktoken-tokenizer.png 1x, \/wp-content\/uploads\/2025\/08\/how-to-use-the-tiktoken-tokenizer.png 1.5x, \/wp-content\/uploads\/2025\/08\/how-to-use-the-tiktoken-tokenizer.png 2x, \/wp-content\/uploads\/2025\/08\/how-to-use-the-tiktoken-tokenizer.png 3x, \/wp-content\/uploads\/2025\/08\/how-to-use-the-tiktoken-tokenizer.png 4x"},"classes":[]},{"id":53774,"url":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/09\/03\/integrate-tiktoken-in-python-applications\/","url_meta":{"origin":53555,"position":1},"title":"Integrate Tiktoken in Python Applications","author":"CPI Staff","date":"September 3, 2025","format":false,"excerpt":"Learn what Tiktoken is and how to use it in Python to count tokens, budget prompts, and chunk text with a practical, step-by-step example.","rel":"","context":"In &quot;AI&quot;","block_context":{"text":"AI","link":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/category\/ai\/"},"img":{"alt_text":"","src":"\/wp-content\/uploads\/2025\/09\/integrate-tiktoken-in-python-applications.png","width":350,"height":200,"srcset":"\/wp-content\/uploads\/2025\/09\/integrate-tiktoken-in-python-applications.png 1x, \/wp-content\/uploads\/2025\/09\/integrate-tiktoken-in-python-applications.png 1.5x, \/wp-content\/uploads\/2025\/09\/integrate-tiktoken-in-python-applications.png 2x, \/wp-content\/uploads\/2025\/09\/integrate-tiktoken-in-python-applications.png 3x, \/wp-content\/uploads\/2025\/09\/integrate-tiktoken-in-python-applications.png 4x"},"classes":[]},{"id":53640,"url":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/08\/18\/build-a-multi-agent-assistant-in-python-with-the-openai-agents-sdk\/","url_meta":{"origin":53555,"position":2},"title":"Build a Multi-Agent Assistant in Python with the OpenAI Agents SDK","author":"CPI Staff","date":"August 18, 2025","format":false,"excerpt":"This post \"Build a Multi-Agent Assistant in Python with the OpenAI Agents SDK\" shows how to build an AI agent that can (a) generate secure passwords, (b) tell the current time, and (c) hand off coding questions to a Python-tutor sub-agent. Along the way, we\u2019ll cover what the Agents SDK\u2026","rel":"","context":"In &quot;AI Agents&quot;","block_context":{"text":"AI Agents","link":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/category\/ai-agents\/"},"img":{"alt_text":"","src":"\/wp-content\/uploads\/2025\/08\/build-a-multi-agent-assistant-in-python-with-the-openai-agen-1.png","width":350,"height":200,"srcset":"\/wp-content\/uploads\/2025\/08\/build-a-multi-agent-assistant-in-python-with-the-openai-agen-1.png 1x, \/wp-content\/uploads\/2025\/08\/build-a-multi-agent-assistant-in-python-with-the-openai-agen-1.png 1.5x, \/wp-content\/uploads\/2025\/08\/build-a-multi-agent-assistant-in-python-with-the-openai-agen-1.png 2x, \/wp-content\/uploads\/2025\/08\/build-a-multi-agent-assistant-in-python-with-the-openai-agen-1.png 3x, \/wp-content\/uploads\/2025\/08\/build-a-multi-agent-assistant-in-python-with-the-openai-agen-1.png 4x"},"classes":[]},{"id":53745,"url":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/08\/31\/understanding-openai-embedding-models\/","url_meta":{"origin":53555,"position":3},"title":"Understanding OpenAI Embedding Models","author":"CPI Staff","date":"August 31, 2025","format":false,"excerpt":"A practical guide to OpenAI\u2019s embedding models\u2014what they are, how they work, and how to use them for search, RAG, clustering, and more.","rel":"","context":"In &quot;AI&quot;","block_context":{"text":"AI","link":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/category\/ai\/"},"img":{"alt_text":"","src":"\/wp-content\/uploads\/2025\/08\/understanding-openai-embedding-models.png","width":350,"height":200,"srcset":"\/wp-content\/uploads\/2025\/08\/understanding-openai-embedding-models.png 1x, \/wp-content\/uploads\/2025\/08\/understanding-openai-embedding-models.png 1.5x, \/wp-content\/uploads\/2025\/08\/understanding-openai-embedding-models.png 2x, \/wp-content\/uploads\/2025\/08\/understanding-openai-embedding-models.png 3x, \/wp-content\/uploads\/2025\/08\/understanding-openai-embedding-models.png 4x"},"classes":[]},{"id":53658,"url":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/08\/20\/building-guardrails-in-the-openai-agent-sdk\/","url_meta":{"origin":53555,"position":4},"title":"Building Guardrails in the OpenAI Agent SDK","author":"CPI Staff","date":"August 20, 2025","format":false,"excerpt":"This OpenAI Agent post \"Building Guardrails in the OpenAI Agent SDK\" will explain how to implement a gurdrail system that protact the Agent from misuse. Table of contentsWhat Are Guardrails?Example: A Python-Only GuardrailIntegrating Guardrails Into the Main AgentTesting the GuardrailWhy Guardrails MatterGuardrails as Part of a Larger Agent DesignConclusion In\u2026","rel":"","context":"In &quot;AI&quot;","block_context":{"text":"AI","link":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/category\/ai\/"},"img":{"alt_text":"","src":"\/wp-content\/uploads\/2025\/08\/Guardrails.png","width":350,"height":200,"srcset":"\/wp-content\/uploads\/2025\/08\/Guardrails.png 1x, \/wp-content\/uploads\/2025\/08\/Guardrails.png 1.5x, \/wp-content\/uploads\/2025\/08\/Guardrails.png 2x, \/wp-content\/uploads\/2025\/08\/Guardrails.png 3x, \/wp-content\/uploads\/2025\/08\/Guardrails.png 4x"},"classes":[]},{"id":56780,"url":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/2025\/11\/10\/security-best-practices-for-azure-ai-services\/","url_meta":{"origin":53555,"position":5},"title":"Security Best Practices for Azure AI Services","author":"CPI Staff","date":"November 10, 2025","format":false,"excerpt":"Practical, step-by-step guidance to secure Azure AI services end to end\u2014identity, networks, data, prompts, and monitoring\u2014so your teams can innovate confidently without exposing your organisation.","rel":"","context":"In &quot;Azure AI Services&quot;","block_context":{"text":"Azure AI Services","link":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/category\/azure-ai-services\/"},"img":{"alt_text":"","src":"\/wp-content\/uploads\/2025\/11\/security-best-practices-for-azure-ai-services-in-practice.png","width":350,"height":200,"srcset":"\/wp-content\/uploads\/2025\/11\/security-best-practices-for-azure-ai-services-in-practice.png 1x, \/wp-content\/uploads\/2025\/11\/security-best-practices-for-azure-ai-services-in-practice.png 1.5x, \/wp-content\/uploads\/2025\/11\/security-best-practices-for-azure-ai-services-in-practice.png 2x, \/wp-content\/uploads\/2025\/11\/security-best-practices-for-azure-ai-services-in-practice.png 3x, \/wp-content\/uploads\/2025\/11\/security-best-practices-for-azure-ai-services-in-practice.png 4x"},"classes":[]}],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/wp-json\/wp\/v2\/posts\/53555","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/wp-json\/wp\/v2\/comments?post=53555"}],"version-history":[{"count":1,"href":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/wp-json\/wp\/v2\/posts\/53555\/revisions"}],"predecessor-version":[{"id":53557,"href":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/wp-json\/wp\/v2\/posts\/53555\/revisions\/53557"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/wp-json\/wp\/v2\/media\/53556"}],"wp:attachment":[{"href":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/wp-json\/wp\/v2\/media?parent=53555"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/wp-json\/wp\/v2\/categories?post=53555"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cloudproinc.azurewebsites.net\/index.php\/wp-json\/wp\/v2\/tags?post=53555"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}