Yes, but don't expect much charge or any at all while gaming.

See this for more details.
Playing With Power: A Look At Nintendo Switch Power Consumption

 

Lol. Ok. Stated specs by a manufacturer are not always accurate so IF that car charger can deliver 2A PER PORT you should hold steady. I noticed you said 2.1A "total" if that is divided between two ports it is going to fall short.

 

You wont fry your Switch unless your car's electrical system is messed up. What you want is 5V 2A (2.1-2.4) out of one port. It will charge decently when the Switch is off and hold steady while gaming.

The BEST way to charge in the car is get a power inverter and use the included AC charger because it will charge at the full 15V. Something along the lines of 200 peak watts (that is how they market them) should be sufficient, though 500 would be more versital for other uses. With that you will be able to game and get the fastest charge possible.

Edit: Because the vehicle outputs 12V an additional solution would be a straight 12V plug USB-C charger but I don't know if something like that exists in a mass produced scale at this point in time. There is a chart in the anandtech piece that lists what the amperage would be at 12V whatever it is it would me more than the Switch would draw which appears to be about 1.9 and would work just as well as the power inverter/AC charger method.

I know it seems a little goofy to go DC>AC>DC but voltage changes sometimes make it the simplest solution. I wear a CPAP at night due to sleep apnea so when camping out in the woods I need a solution. My CPAP is 24V DC so I lug around a big 12V marine deep cycle battery, clip a 500 peak watt inverter to it and plug my CPAP power brick to that that converts it to 24V DC.

Somewhere in the Wii U section is a post where someone did something similar to make the Wii U portable for camping. Lol.