Mr Kawczynski said it was important that the two parties can reach an agreement after the election saw Theresa May’s party fall short of a Commons majority.
The DUP has regularly courted controversy in its near 50-year history for its views on issues, which include opposing same sex marriage.
Mr Kawczynski, who announced he was bisexual in 2013, said: "I know members of the DUP very well. I have worked with them in the past.
"We will be meeting the PM today to hear her views.
"We will try and have an agreement with the DUP and see whether they will support us with their votes in the House of Commons, which is very important.
"But we ought to have some kind of agreement with the Lib Dems which supports us on some issues which are socially liberal.
"Many people in the Conservatives have different views. It is a broad church. People in the party voted against same sex marriage. To castigate a party is irresponsible.
"We have to work together whatever the issues are to provide a government and form a government which is stable. No one wants another election."
Scottish Conservative leader Ruth Davidson has been told by the prime minister that any Conservative deal with the DUP will not affect LGBTI rights.
Ms Davidson who is gay, plans to marry her partner in the near future.
The DUP opposes same-sex marriage and Northern Ireland the only part of the UK where it is not legal.
Ms Davidson said she had been given an assurance that gay rights would not be eroded in return for DUP support.