Parallel Processing
Making serial requests to the API can be slow due to network overhead: each request must finish before the next one is made. To maximize throughput, you can process requests in parallel with multiprocessing or asynchronous requests. In this guide, we'll show how to make many concurrent requests with asyncio and aiohttp in Python.
If you want to maximize throughput for a one-off job, you might also consider using our Batch endpoint to classify many texts in a single request, and get back a single file with all the results.
Example Code
import asyncio
import aiohttp
url = "https://api.trytaylor.ai/api/public_classifiers/predict"
model = "topics"
api_key = "xx-your-api-key-here"
async def classify_batch(texts: list[str], session: aiohttp.ClientSession):
async with session.post(
url,
headers={"Authorization": f"Bearer {api_key}"},
json={"model": model, "texts": texts},
) as response:
data = await response.json()
print("batch done!")
return data
async def classify_many_texts(texts: list[str]):
# maximum supported batch size is 20
batches = [texts[i : i + 20] for i in range(0, len(texts), 20)]
async with aiohttp.ClientSession() as session:
tasks = [asyncio.ensure_future(classify_batch(batch, session)) for batch in batches]
gathered = await asyncio.gather(*tasks)
results_list = []
for res in gathered:
if "results" in res:
results_list.extend(res["results"])
else:
print("Uh oh!")
return results_list
def main():
# assume each line is a text we want to classify
texts = open("input.txt").readlines()
classifications = asyncio.run(classify_many_texts(texts))
with open("results.json", "w") as f:
f.write(json.dumps(classifications, indent=2))
if __name__ == "__main__":
main()