Whitelisting IP for a specific port using rich rules in firewalld
firewall-cmd --add-rich-rule='rule family="ipv4" source address="x.x.x.x" port port=22 protocol=tcp accept'