https://netmaker.org logo
Join Discord
Powered by
b
Hi guys -- came across Netmaker when researching about MeshVPN options out there. As far as I understand, there are 2 components -- Netmaker Server and Netclients. What is the suggested way of setting up the Netmaker Server: - in an EC2 instance in the cloud? - inside a K8s cluster? - what if I want to setup multiple Netmaker Servers for redundancy? My use case is that I have a fleet of M1 Mac devices out in the wild spread throughout the globe and all of these will become clients in a Mesh Network setup Thus, the server (I presume) has to sit outside of this fleet (see the options above) Let me know if this is the correct approach towards using Netmaker. Also, is 
netclient
available as a single binary?
b
Netmaker server should be installed on a machine with a static public ip. A vps from any of the cloud providers is ideal. Netclient is a single binary. There is also a service file to control starting and stopping of the daemon. The service file is embedded within the binary and will will deployed to correct location by running./netclient install
b
I see, so, is downloading of 
netclient_darwin_arm64
from https://github.com/gravitl/netclient/releases/tag/v0.18.5 is enough? Because it's a binary I ask since there is also this file called 
Netclient-M1.pkg
b
After downloading the binary, run ./netclient install
b
Also, does the node/client have to have 
wireguard
installed before?
b
joining a network will not result in anything showing with wq show unless the netclient daemon is running
netclient does not depend on wireguard-tools
b
I see
so even if I don't have 
wireguard-tools
installed, I can still run 
sudo wg
after 
./netclient install
right?
b
wg is part of wireguard-tools. Not needed for netclient to operate but wg show won't work without it
Fyi, there is also a homebrew package for netclient
b
If I have one server and one client and I join the client then 
sudo wg
on server has two entries while 
sudo wg
on client has just 1 entry is this expected?
actually wait
now it's fixed
2 entries in both
thanks
b
If the daemon on the client is not running, yes expected behavior
b
basically, earlier, I did 
netclient join -t TOKEN
before 
./netclient install
and that was the source of all misery
thanks a lot @bored-island-21407
one quick last question perhaps -- where can I see the daemon running?
I am on an M1 mac
and my server is a EC2 in the cloud
b
The join command updates config files, the setup of the wireguard interface is handled by daemon
b
I see, and wireguard wasn't setup because of not running 
./netclient install
b
Look at your running services or use pgrep -a netclient
b
also, if I have just one server in the EC2, how many nodes can it handle in the mesh network
what if I want to setup multiple servers?
b
stress testing of server is ongoing.
You can have multiple servers if you like. A client can join networks on multiple servers
b
is there any guide around setting up multiple servers
how they communicate with each other etc.
this is for redundancy of course
b
Look in docs for HA (high availability)
There is a link to docs in UI
b
For HA -- we have options of Kubernetes and multiple EC2s right?
b
I think it is k8, not 100% sure
b
ok, will check, thanks a lot
what is the difference between
register -t TOKEN
and 
join -t TOKEN
?
b
join is an alias of register (used to be different)
b
one last question
when I do
./netclient_darwin_arm64 install
I get
Copy code
dragonfruit@Dragonfruits-Mac-mini Downloads % sudo ./netclient_darwin_arm64 install
open ./build/appicon.png: no such file or directory[netclient_darwin_arm64] 2023-04-12 17:10:58 setting OS 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting version 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting netclient hostid 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting name 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting macAddress 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting wireguard keys 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting wireguard interface 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting listenport 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting proxyListenPort 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting MTU 
[netclient_darwin_arm64] 2023-04-12 17:10:58 setting traffic keys 
[netclient_darwin_arm64] 2023-04-12 17:11:00 open /usr/local/bin/netclient: no such file or directory 
[netclient_darwin_arm64] 2023-04-12 17:11:00 error installing daemon open /usr/local/bin/netclient: no such file or directory
this is on a fresh new device
b
You should change the name of binary to netclient
b
/usr/local/bin/netclient: no such file or directory
basically
I still get the same
even after renaming
what made it work earlier for me is that I also installed via 
Netclient-M1.pkg
but I was trying without it
b
change netclient_darwin_amd64 to netclient and rerun ./netclient install
b
yes I did that, seems like 
bin
folder didn't exist for me, running again
thanks, worked
@bored-island-21407 I joined a new node again and while it appears in 
sudo wg
for both server and 1st node the 
ping
to it doesn't work and I get 
Request timed out
but all 3 nodes (1 server + 2 client) appear as Healthy in UI
10.101.0.1 is server 10.101.0.2 is 1st client 10.101.0.3 is 2nd client
b
When you run wg show is a handshake showing? Rx and tx show positive numbers
is there a firewall on node3?
b
so 
ping 10.101.0.3
from the machine with 
10.101.0.2
doesnt work but it works when I do it from server aka 
10.101.0.1
so yeah, handshake is not present in 
10.101.0.3
entry when doing 
sudo wg
from client with 
10.101.0.2
b
Is node3 behind NAT?
b
I am not sure, I have asked, btw, this node3 doesn't have 
wireguard-tools
installed
and ping to node3 from server (node1) works but ping to node3 from node2 doesn't work
b
node2 and node3 are on different lans or the same?
b
different wifi different cities
b
setup server as relay and relay node3
b
why is this needed?
also, is there a doc on how to do it?
b
node2 and node3 are probably both behind NAT, depending upon the type of NAT they sometimes cannot communicate directly
b
I have UDP Hole punch enabled btw, but I guess that doesn't help?
and wouldn't relay be slower?
b
On the host tab, select the server host where you can set it to be a relay
a relay adds an extra hop so it will be slower, how much depends on routes between all the nodes
b
so I go to hosts, click on server (node1) and turn on 
is relay
?
maybe I have to turn on 
is relay
for 
node3
and not 
server (node1)
right?
b
server should be relay and node3 should be relayed
b
ok, I did that, still same issue cannot ping node3 from node2
Request Time out and No entry for 
latest handshake
when doing 
sudo wg
on node2
b
It sometimes takes a little while for relay routing to settle down
b
is the same problem experienced with other tools such as Zerotier, Tailscale etc.?
PS -- I haven't tried them
and am new to this domain
b
Yes, and they all have different methods for getting around the problem
b
I see, and I guess I came across the benchmarks article on medium do those benchmarks incorporate relaying?
b
Which benchmarks?
b
b
I am not sure whether relays were included in those tests
b
I see, still waiting for handshake to appear btw Should node2 be relayed as well? I ask since it could be a similar issue of node3 not being able to access node2
b
If you want to understand the problem better, read up on STUN and TURN
You should only have to relay one of the nodes
b
and if relaying is turned on, does it use relay all the time OR priority is still given to UDP Hole Punch / other ways
b
When relaying , all traffic to,from the relayed node goes through the relay
b
also, still waiting for handshake, any ETA available? and hmm, this relay stuff has to be manually turned on after looking into issue? can it not be automatically figured out?
I ask since if there are 100s of nodes
then manual inspection would be cumbersome
b
It is being worked on
it is complicated to get correct. Especially preventing unneeded relays
b
ok, I am still waiting for relay stuff to come up
btw @bored-island-21407 sorry for tagging again node1 can ping both node2 and node3 node2 cannot ping node3 but can ping node1 node3 cannot ping both node1 and node2
b
there must be a firewall that prevents node3 from communicating
it is strange that node1 can ping node3 but node3 cannot ping node1
b
this was after turning relay on
I have disabled, lets try again
seems like the latest handshake was 22 mins ago for node3
and now node1 cannot ping node3 too
b
can you delete node3 and then rejoin?
b
ok
I just made it rejoin, seems like handshake entry for node3 is present in server (node1) but not in node2
but then, the handshake entry present for node3 in server (node1) doesn't update
I mean, neither node1 or node2 can contact node3
relay is disabled btw
b
for the relay to work; the relay and relayed need to be able to communicate
so in your case node1 and node3 need to communicate or relay won't do anything
b
if node2 pings node3, request timed out if node1 pings node3, the terminal has not output and hangs aka something like this
Copy code
ubuntu@ip-172-31-94-115:~$ ping 10.101.0.3
PING 10.101.0.3 (10.101.0.3) 56(84) bytes of data.
b
what is output of ping -c 4 10.101.0.3
b
Copy code
PING 10.101.0.3 (10.101.0.3) 56(84) bytes of data.

--- 10.101.0.3 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3052ms
this is from server (node 1)
b
can you ping the publicip of node1 from the server?
b
this is from node2
Copy code
PING 10.101.0.3 (10.101.0.3): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2

--- 10.101.0.3 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
> publicip of node1 from the server? server is node1 only you mean something else I guess?
b
node3
typo
b
so its 171.76.80.43 according to google search of whats my IP
so it's the same
Copy code
PING 171.76.80.43 (171.76.80.43) 56(84) bytes of data.

--- 171.76.80.43 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3059ms
b
there has to be a firewall in front of node3
b
I see, and is relay able to solve this?
not really, right?
b
depends on firewall settings
if it is blocking everything, then there is not much you can do
b
is the same issue faced by Zerotier et al i.e. other tools as well? aka no workaround in such a case?
b
wg has to establish a handshake; until that is done, not much any tool can do about it
b
but handshake for node3 was present in 
sudo wg
output of 
node1
, but not in output of 
node2
b
that is not unusual if node3 and node2 are both behind NAT.... node3 can see node1 because node1 is a vps that does not have NAT
b
but yeah, handshake for node3 doesn't update, I mean, the timestamp doesnt update
b
the persistent keepalive is 20 seconds, correct?
b
yes
Copy code
interface: netmaker
  public key: wKLx81Cv6Fx5WvWl8xtV+2jG55zpw+zNCgG1nOpUORE=
  private key: (hidden)
  listening port: 51821

peer: sopYNdvjx5ZTQy9feW36oEkaLwR3ZCHKaSdycAMSCRs=
  endpoint: 182.69.183.89:7543
  allowed ips: 10.101.0.2/32
  latest handshake: 20 seconds ago
  transfer: 26.16 KiB received, 19.18 KiB sent
  persistent keepalive: every 20 seconds

peer: K1/sc53TDIp786+jzAY0wl0evSQCbYTPj3ab3n2dAgk=
  endpoint: 127.0.0.1:53751
  allowed ips: 10.101.0.3/32
  latest handshake: 17 minutes, 40 seconds ago
  transfer: 212 B received, 29.89 KiB sent
  persistent keepalive: every 20 seconds
this is output from node1
b
what is output from node3
b
node3 doesn't have 
wireguard-tools
installed
b
ok
b
and doesn't have 
brew
also installed + some other person has control over it so I have to ping him to run commands etc 😅
3 Views
PreviousNext
Powered by