A bit of background we have an original site with the following local network:-
Site1
Network: 10.1.3.0/24
This network has the server locations.

A small distance away is another building we have moved into. there was already a fibre connection, so we have utilised that with some fibre to ethernet adapters.

So, we have setup two Draytek Vigor 2860's
Site 1:
LAN1
IP 10.1.3.3/24
DHCP 10.1.3.10 to 10.1.3.20
WAN2 
IP: 194.184.180.2/30
GW: 194.184.180.1 (Site 2 WAN2)

Site 2:
Has VLANS configure for different applications, trusted, security, management
VLAN1 10.180.1.0/24
VLAN2 10.180.20.0/24
VLAN3 10.180.40.0/24
VLAN4 192.168.50.0/24
WAN2 
IP: 194.184.180.1/30
GW: 194.184.180.2 (Site 1 WAN2)

Currently I'm testing to get Site 1 to communicate with Site 2 VLAN 2 10.180.20.0

I thought setting up matching static routes would work.

Site 1
Destination : 10.180.20.0
Subnet: 255.55.255.0 /24
Gateway: 194.184.180.1
Interface: WAN2

Site 2
Destination : 10.1.3.0
Subnet: 255.55.255.0 /24
Gateway: 194.184.180.2
Interface: WAN2

I enabled pings on external ports

I have laptops placed at either side getting the following IP's
Site 1 Laptop 10.1.3.10/24 GW 10.1.3.3

Site 2 Laptop 10.180.20.10/24 GW 10.180.20.1

Each laptop can ping up to the WAN2 interface port of the other sites router.

but alas just running tracert gets to the WAN2 interface of the other router and gets no further.
I'm sure I'm missing something simple.

there was already a fibre connection,

Do you mean a fibre connection between buildings (rather than a fibre WAN connection to an ISP at the new building)?

Do you have the WAN2 ports of the routers connected via that fibre?

Yes there was just a fibre connection. So the ports are directly connected.

I just setthe WAN ports up to mimic a public IP

Well to start with, if you are connecting WAN ports like this I think it would be better to use private IPs. Save confusion.

Just a tiny little subnet for the link, anything you are not using elsewhere, you could perhaps use something in the 172.16/12 block.

And also, using the WAN ports in this way, the routers will be performing NAT (also firewall), which is likely preventing your pings getting 'inside' the networks... Hence you can only ping up to the WAN interface.

You could use Open Ports + Appropriate Firewall Rules, an IP Routed Subnet (No NAT), or perhaps look into using a VPN over the P2P link you have? (Could set the encryption very low to reduce overhead if needed, since its a private link anyway)?