Yes, you should be able to do this. This will work much like HDMI selector switches which allow combining multiple HDMI feeds to drive a single HDMI input on a TV. I have a 4x2 switch in my setup (4 HDMI in, 2 HDMI out) and it works fine.
Obviously, both TV's will see the same channel and picture.
