Computer slows down massively when two other monitors are hooked up to it (even when the two other monitors are idle). Is the GPU a possible culprit?

My computer slows down massively when two other monitors are hooked up to it. You'd assume that this would be caused by a memory/processor issues but here's the thing - the computer slows way down even if, immediately after hooking up to the other monitors, the monitors are left idle. Is it possible this is a GPU issue? Is it even possible for the GPU to make a computer run more slowly if too much is demanded from it? It seems like if too much were demanded from the GPU, the image quality - and not computer performance - would be affected. I don't think this is a memory or processor issue (my RAM and the CPU aren't being maxed out when I experience said performance issues). Is it possible this is a GPU problem?