[torch.compile] hide slicing under custom op for inductor

[youkaichao]

see pytorch/pytorch#131192

when inductor sees a view being mutated, it will copy the tensor.

hiding the slicing operation under custom op solves the issue for inductor.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[youkaichao]

cc @laithsakka

[youkaichao]

after this pr, the major bug of inductor ( it tries to copy kvcache) should be fixed.

however, there are still some inefficiencies w.r.t. inductor. it cannot deal with inplace custom op well.

[bnellnm]

Thanks for finding and fixing this!

[laithsakka]

This is fixed on pytorch 2.5 by setting TORCHDYNAMO_AUTO_FUNCTIONALIZED_V2=1

[laithsakka]

can you elaborate more on "there are still some inefficiencies w.r.t. inductor. it cannot deal with inplace custom op well."

[youkaichao]

you can take a look at the log when you run with inductor:

# GPU blocks: 790

when I turn on inductor, the blocks become smaller, which means inductor takes more memory and we have less memory for the kv cache.

[laithsakka]

takes more memory and we have less memory for the kv cache.

I see i will create an issue to revisit this, for the models i ran inductor on , it was slightly better than not using torch.compile. I will sync with you on that
pytorch/pytorch#136269