Lyell Hintz patreon

StreamDiffusionTD 0.2.3🚀🖼️ [ Changelog ]

🕑 Added 2024-10-09 01:18:01 +0000 UTC

Comments

vaishnavi shevde

hey i am having the same problem.... Were u able to resolve it and if yes how ?

AZIZA ABDELHAK

I have a problem is : TensorRT is not found in the virtual environment

Daniel Gary

Eagerly waiting for this week's update!

Christopher Lensing

HeyHo, as I start a new realtime AI project these days: any news on the new version. I am really curious about what you have been cooking and if it's worth to wait some days before I start.

Hardik Dhingra

Also, can I get a discord invite?

Hardik Dhingra

Hey! been trying the streamdiff tool. It's pretty amazing. got an issue though. tensorrt doesn't work after changing the resolution. default 512*512 resolution works fine though. is there any workaround for this?

mr Zpacely

did u ever get this resolved?

Stefano Settembrini

Hey guys, is there a video about huggingface / other models / lora / checkpoints downloaded from different sites (civitai, ecc) ?? I'm really curious about experimenting, and the base SD model from the component works but I'd love to customize more with other models

seboya

Is it possible to choose which GPU it uses if I have more than one

Daniel Gary

Wonderful! Looking forward to it!!!

Daniel Gary

PERFECT!!!

dotsimulate

hey ! ive got a new version coming out this week. And tensorrt supports up to 1024 per side.

dotsimulate

this week. putting final touches / release notes together currently.

dotsimulate

supporting python 3.11.9 on the next version (and still compatible with 3.10.9) cuda support stays on 11.8/12.1 at the moment.

dotsimulate

Yes ive got an update coming this week. Just working on documentation and final fixes for apple support.

Sergej G

Was wondering if there is going to be an update for compability with new cuda and python versions?

Daniel Gary

Is this project abandoned? If not, when is the next release?

Leon Krykhtin

"Add Python, CUDA, and Git to your system's PATH during installation." What does this mean? I just installed it normally to my computer system. What else do i need to do? I didn't see any questions like that when i was installing all three components... Please help me

Dharuma Trejo

Hello :) Where can I install new models?

Oscar Ingham

Loving this component so far! Great work. I'm interested in training a custom LoRA - any advice on how to get started? Also, can you please share any other LoRA known to work well with this TOX?

Nick Bogannam

I would check your huggingface folder location on the installation page.

Miranda

Hello, I am trying to change the 'SD model id/ path' on the stream setting page to that of a model stored locally on my computer, however it is unable to run giving me an error that it needs to be a URL path to a huggingface model. Any direction of this would greatly be apreciated

Daniel Gary

When is the new version coming out? What's the newest and highest resolution model that I can run with the current (and upcoming) release, with tensorrt enabled?

Pascal P

it works now but i loocking for a solution the model is taken so many GB when i hit pulse "startstream"

Daniel Gary

What's the newest and highest resolution model that I can run with the current (and upcoming) release, with tensorrt enabled?

Daniel Gary

sdxl turbo did not work for me with tensorrt, normally it works.. Also, almost none of the models that I get from civit.ai work - they either throw and error or generate bad output

dotsimulate

which new models? some recent bigger image models have a different architecture and are quite heavy - like sd 3.5 and flux which won't be supported in next release. but things are changing always and im keeping my eye out.

Daniel Gary

New version? We can't use new models with this one. Will the new version support new models?

Roman Rusavsky

Before installing tensorRT try to Pulse Manual Activate VENV and execute the following commang in terminal: pip install --upgrade huggingface_hub==0.24 after that tensorRT will install ok.

Andrea

Hi Mate! Thank you for version 0.2.3. However, I'm encountering an issue with TensorRT—it seems to be missing in the virtual environment. I need to resolve this. Do you have any tips or tricks to fix it? Please let me know. Thanks! have a nice day!

구도의밭

Thank you, artist. To use the 0.2.3 version, do I not proceed to the Cuda 12.6 version, but to use the latest version, do I need to change it to 12.1 and install it? Or will it proceed with 12.6 version without any problems?

Lukas Matthys

Hello, this is a wonderfull TOX! Quick question: Is it possible to use the new SD 3.5 large turbo model?

Jeroen Hofs

I was stuck here too. The .tox install pulses seem to use the latest python version installed. What finally worked for me was uninstalling everything leaving only 3.10, then the 2. Install [ venv + all req ] step completes without errors for torch and then the 3. Install TensorRT will complete. I also opened the venv in Powershell and ran "pip install tensorrt-cu11 tensorrt-lean-cu11", to see if I needed to install TensorRT according to this page: https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html but I'm not sure if that is related

Christopher Lensing

Great news. I played around with SD TD after quite some time the other day and was wondering if there is still something going on with this tool, as it is so very, very cool to play with. No intention to push you included. Stream Diffusion itself seems to be out of development for month now and I was wondering if everyone just left the party for some other stuff....

dotsimulate

Yep! Ive been working on revamped mac support and embeddings for streamdiffusion (coming in about a week once I can record a video). Mostly just been busy due to recently moving apartments and traveling most of November/December. also have been developing a full new operator class that has been taking more dev time.

monokuromasu

I appreciate this and your work so much. Thanks for giving us this, Dot.

Christopher Lensing

Is this actually still a thing. Do you still work on this?

dotsimulate

hey ! this link should work - https://discord.gg/mk2Tc9wM

Keli'i Centurion

Does anyone know how I can join the Discord Group? I paid for patreon so that I could use Stream Diffusion but it wont even install because of something called TensorRT. I googled it but now I have to know how to use coding to install from github. I dont understand any of it.

Andrei Ionut Apopei

When I pulse Start Stream I got the following error: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /CompVis/stable-diffusion/main/configs/stable-diffusion/v1-inference.yaml (Caused by NewConnectionError(': Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))

Mattia Piatti

Ho lo stesso identico problema

joão penna

I'm trying to use this version but the only way can get it to work is using 144x144 pixels, and tensorrt is kinda weird, when it works it says that it does not have enough memory, when it doesn't the images start to generate (just in 144x144px)

Alessandro Sasso

Hi everyone, I'm trying to download streamdiffusion I'm just missing the tesorRT, the problem is that when I try to download it it gives me this error: from huggingface_hub import HfFolder, cached_download, hf_hub_download, model_info ImportError: cannot import name 'cached_download' from 'huggingface_hub' (C:\StreamD\StreamDiffusion\venv\lib\site-packages\huggingface_hub\__init__.py). Has this happened to anyone? Do you know how to fix it? Because without downloading this it doesn't even let me start the prompts. If I try to start one it still tries to download the tesorRT without success.

Justin

how do you fix this issue ?

Adrian Stein

hey guys! i was using the operator pretty smoothly, but it seems to be stuck when building the tensorrt engine. I know it takes a while but ive waited now way longer than i used to. tried building an engine for 512 x 512 and then 960 x 512 but no luck in either.

Christopher Davenport

same issue

Tony Q

Seems like Hugging Face might change API links. There is problem then installing TensorRT on a new location, (ImportError: cannot import name 'cached_download' from 'huggingface_hub' , Did you mean: 'hf_hub_download') also there is new error, happend when streamdiffusion install ( subprocess-exited-with-error × git clone --filter=blob:none --quiet https://github.com/dotsimulate/StreamDiffusion.git ) - when you install from a this tox into new location also.

Serhii Panin

The same issue with 0.2.3

Daniel Heitmüller

Hellooo :) How can i add a negative prompt? :)

Yiwen Wang

Hi there! Sorry if this is a silly question, but how can I output an image with the same dimensions as my input? My output size is currently fixed at 512x512 px, but I’d like to output an image at 1920x1080.

A Olis

Hey there - how is this going? Would love to get this working on Mac. Hope all is well!

Josh Mutscheller

Has anyone figured out how to enable the safety checker? Trying to keep it very strict SFW

Matias Piipari

I'm failing at getting StreamDiffusionTD to behave with either SDXL 1.5 or SDXL-Turbo, with xformers. I managed to patch StreamDiffusion myself for a Linux build of it, and it works over there, but it looks like the LCM-LoRA doesn't get loaded (I also tried giving a custom one): I need a very large step count for the model to produce an image corresponding to the input image, and it doesn't appear to follow the prompt. Swapping back to SD-Turbo makes it all work again. Are there instructions somewhere for how to configure the node to work with SDXL 1.5 or SDXL-Turbo?

Ada Kanlıbaş

Yeah I am sure, I tried redownloading aswell but every time it connects to server and when I click streamdiffusion, it crashes. I am happy to send you a video from DM if you have free time. Ive already sent you a text

maria carame lucenilla

Hi! have you been able to do this? I have the same problem, with a 4060 I only have 2 or 3 fps the nvidia has 8gb of ram it uses 3gb

dotsimulate

hey ! which model are you running? sd-turbo is 2.1 based and won't be compatible with 1.5 loras. if not sd-turbo and using a 1.5 model if you can send me links or something to the base SD model as well I can try and recreate.

dotsimulate

Hey Ada. I am not quite sure. Is there any chance that your project is in the OneDrive location on your pc?

Lorenzo HMZ

did any of you succeed with making a LoRA work? Could you please link it to me and tell me your specs? 🙏🏼 thanks

Lorenzo HMZ

Does anyone have issues loading any kind of LoRA? I'm running on a Windows 11, RTX 4080 SUPER, python 3.10.9, cuda 12.1 a SD1.5 model with 3 steps and V2V. Every LoRA has a slightly different type of error; this LoRA here [https://civitai.com/models/109043/skin-and-hands-malefemale-from-polyhedron] should be able to support SD1.5, but I still get a long error starting with this: Error loading LoRA 'polyhedron_new_skin_v1point1': Error(s) in loading state_dict for UNet2DConditionModel: size mismatch for down_blocks.0.attentions.0.proj_in.lora_A.polyhedron_new_skin_v1point1.weight: copying a param with shape torch.Size([128, 320, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 320]). Anyone that could help?

Ada Kanlıbaş

I was running streamdiffusion smoothly but fow some reason it just started to crash whenever I start streaming. It connects to the server without problem but for some reason every time I press stream, it crashes and says it is due to touchdesigner/project1/streamdiffusionSD/numphy_share_out. Is it only happening for me, or do I do something wrong???

Julii Haid

Hi Dot i want to ask what is this error, i change pc, try to insall 0.2.3 and this happend " cloning into 'C:/ StreamDiff/StreamDiffusion'... error: RPC failed; curl 92 HTTP/2 stream 5 was not closen cleanly : CANCEL (err 8) error: 6465 bytes of body are still expected fetch-pack: unexpected disconnect while Reading sideband packet fatal: early EOF fatal: fetch-pack:invalid index-pack output "

Armin Schl

I'm running streamdiffusion on AWS - is it possible to use that server inside touchdesigner?

RocketShip Trips

Well I tried setting up a new project and had to do tensorrt optimisation again and now it's easily hitting 24fps so I'm happy, also haven't encountered any crashes so pretty happy so far. But I admit I did a bit of a dumb and it seems resolume was ALSO running on the rtx and that was supposed to run on the quadro. But what I get from your answer is that 8 fps is normal for an rtx3090?

James Clarke

tensorRT doesnt work with contronet, if you have controlnet active TensorRT will be automatically diabled and youll onoly get the 8fps. Look at the CMD window when its loaded and it will tell you

RocketShip Trips

So I have some performance related issues. Running v2.3 and latest td on RTX3090 and getting only 8ish fps regular and maybe 5fps in v2v/controlnet (dont know if these are normal fps numbers but I think thats pretty low for rtx3090). When I enable tensorrt I got 15fps (although it looked like far more) so great! But ONLY the first time, now every other time after I start it WITH tensortrt enabled I only get 8ish again. Weird thing was, I didn't even close td, I just stopped the stream and started it again later without changing anything. Since then I obviously did reboots and such to make sure that wasn't it, ran it without any other apps open (although when I got higher fps I had a lot of stuff open and running). Should I remove rensorrt and redo the optimizations or just redo the whole streamdiffusion installation? What is the best approach?

Mirko Ditroia

Is it no longer possible to control the Denoise Schedule of Input 1?

Jeff Fay

does this work for mac?

dotsimulate

Hey. If you have troubles with the python window, there is a debug toggle in the settings 2 page that will pause the cmd window on any errors, allowing you to see the issue. Best place to check would be the discord once you know what is going wrong. Check or post to the troubleshoot forum !

Dan Fallen

seems to be from when installing Tensor getting an error about module streamdiffusion not found

Dan Fallen

I'm getting this error "TensorRT not found in the virtual environment" even though I installed it from the operator

Guglielmo Guarnera

triton is only on linux version of streamdiffusion, that is not causing your problem

giang.it

Hi is there any document on how to download hugging face models and use only the newly downloaded models for StreamDiffusionTD?

Dr. Mindset

with StreamDiffusionTD 0.2.3 I'm getting this error: A matching Triton is not available, some optimizations will not be enabled. Error caught was: No module named 'triton' I'm not sure however if this is causing the traceback error that is causing the larger failure which is a show-stopper for me.

Cate

Hello! What’s the best way to troubleshoot? Start Stream brings up a terminal window for half a second before the window closes and status remains Not Streaming.

eclectictasteofbot

hello, did you figure out how to increase it? I have 16gb too and it shows that currently only 1gb is utilized. thanks!

Ethan Gates

Trying to get this working on Mac and keep running into this error: Preparing Stream... Process Process-4: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/Users/ethan/SD223/StreamDiffusion/streamdiffusionTD/main_sdtd.py", line 369, in image_generation_process pinned_input_buffer = torch.empty((height, width, 3), dtype=torch.uint8, pin_memory=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Need to provide pin_memory allocator to use pin memory. Please help

Dromp Alex

Hello, there is an issue with memory allocation if I try to use any other nvidia stuff, like bg removal, stream diffusion will lock out the node, mediapipe gpu will act the same if I try segmentation. I think this might be an issue for the streamdiffusion git repo. I got a 4070ti and the gpu memory is around 6.3/12gb with the tox running an sdxl turbo with control net at 512x512px. cannot create Cuda stream - an illegal memory access encountered.

Andres Felipe

Hello, everything is working perfectly, thank you very much. However, I have a question: how can I increase the amount of VRAM I am using? I have a 4060 with 16GB, but I am currently only utilizing 4GB to run the Tox

Sebastain Graf

I Love this update, just hit 56fps on 512x512 SDTurbo with TensorRT Thank you! so Incredible! Please support FLUX.1-schnell soon, this would really take my projects to the next level. and it seems the 4090 has enough headroom for it! Error : KeyError: 'time_embed.0.weight' Failed to load C:\Projects\StreamDiffusionTD-0.2.3\models\Model\FLUX.1-schnell\flux1-schnell.safetensors / SD1.5 + LCM LoRA model from both local and Hugging Face due to: 'time_embed.0.weight'