[Frontend] Added chat templates for LLaMa4 pythonic tool calling

[yeqcharlotte]

Summary

Fixing tool calling related chat templates for llama4. LLaMa4 followed similar pythonic format. Co-authored with @wukaixingxp.

Tool call messages and tool response messages should end with <|eom|>. <|python_start|> and <|python_end|> are not very consistent in the output with current checkpoint. Some details in: meta-llama/llama-stack#1886.

Unit tests

pytest -s -vv tests/tool_use --models llama4 --extended
...
======================================================== 149 passed, 50 skipped in 344.22s (0:05:44) =========================================================

Integration tests

with llama-stack
server:

export PORT=8081; export LLAMA_DIR=~/local/checkpoints/Llama-4-Scout-17B-16E-Instruct; VLLM_DEBUG_LOG_API_SERVER_RESPONSE=1 VLLM_TRACE_FUNCTION=1 VLLM_USE_V1=1 SAFETENSORS_FAST_GPU=1 wp vllm serve $LLAMA_DIR -tp 8 --host :: --port $PORT --served-model-name default --max-model-len 32768  --enable-auto-tool-choice --tool-call-parser pythonic --chat-template examples/tool_chat_template_llama4_pythonic.jinja

llama-stack client

export INFERENCE_MODEL=default; export MODEL=default; export VLLM_URL=http://localhost:8081/v1; export VLLM_MAX_TOKENS=20000; pytest -s -v tests/integration/inference/test_text_inference.py --text-model $MODEL --stack-config remote-vllm --vision-model=$MODEL

FAILED tests/integration/inference/test_text_inference.py::test_text_chat_completion_with_multi_turn_tool_calling[txt=default-inference:chat_completion:multi_turn_tool_calling_03] - assert '123' in "<|python_start|>addproduct(name='widget', price=19.99, instock=true, tags=['new', 'sale'])<|python_end|>"
FAILED tests/integration/inference/test_text_inference.py::test_text_chat_completion_with_multi_turn_tool_calling[txt=default-inference:chat_completion:multi_turn_tool_calling_04] - assert 'no' in '<|python_start|>[get_event(date="2025-03-03", time="10:00")]<|python_end|>'
FAILED tests/integration/inference/test_text_inference.py::test_text_chat_completion_with_multi_turn_tool_calling[txt=default-inference:chat_completion:multi_turn_tool_calling_05] - AssertionError: assert '1000' in '<|python_start|>[getmonthlyexpensesummary(month=1, year=2025)]<|python_end|>'
================================================================================================ 3 failed, 20 passed, 3 warnings in 53.22s ================================================================================================

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[yeqcharlotte]

cc: @maxdebayser @K-Mistele @robertgshaw2-redhat @DarkLight1337

[houseroad]

Looks good. So the failed unttests in the llama-stack are expected?

[yeqcharlotte]

Looks good. So the failed unttests in the llama-stack are expected?

llama-stack implementation hasn't passed all of them yet. it does pass 1 more test than vllm now. multi-turn tool calling is a bit flaky and failing for many llama4 inference providers too. @wukaixingxp and team are still actively investigating.