Mr Kawczynski is due to address the National Conservatism conference in Rome on Monday. But he said he would now consult with whips before making his final decision, following criticism in a national newspaper.
Appearing on the platform will be the Matteo Salvini, former Italian deputy prime minister and Hungarian prime minister Viktor Orban, and Le Pen's granddaughter Marion Marecha, a former French MP.
Professor Rafał Pankowski, a co-founder of the anti-fascist organisation Never Again, voiced his concerns about Mr Kawczynski's attendance.
"The event brings together radical conservatives, right-wing populists and neo-fascists," he said. "It is an example of growing trans-national cooperation of authoritarian forces.
“It is important to draw a clear line between democratic political leaders and the far right, but the distinction is becoming blurred as a result of actions of conservatives like Kawczynski. Through rubbing shoulders with the far right, they legitimise xenophobia, which is a growing global problem."
Margaret Hodge, parliamentary chairman of the Jewish Labour Movement, called on Prime Minister Boris Johnson to stop Mr Kawczynski attending the event.
“Johnson does not miss an opportunity to criticise Labour’s anti-semitism," she said.
"While I have agreed with him at times on that, it would be intolerable if he did not stamp out the attendance of one his senior MPs at a conference that will promote racist views."
But Mr Kawczynski said the event would include about 40 speakers spanning a broad range of opinion, including a former speechwriter to Margaret Thatcher and an ex-Polish ambassador.
He said the conference was to discuss national sovereignty, a subject close to his heart due to Poland's experiences during the Communist era.
"Admittedly, Messrs Orban and Salvini are not to everyone’s taste. I imagine that on a great many issues even I would disagree with them," he said.
"But I would absolutely defend their right to speak at a conference on the subject national sovereignty, the very thing they have pledged to defend and which accounts for their popularity with voters."