This is not difficult.
Do you want to have Indesign automatically switch the opening page of a chapter to left (verso) or right (recto) based on what side of the spread the previous chapter ended on? If so, you should just have the Book feature using auto page renumbering. If you want to manually decide which side of the spread each starts on, just set the section opener page # for each file as the page # you want to use, and tell the Book feature not to renumber the pages.
You can control this from the Book panel, whichever way you choose to do it.
However, your inside/outside margins may not come out correctly when the pages are switched from verso to recto, depending on how they were set up. In InDesign CS6 and CC, you can use a primary text frame on the Master page to help this work better for the main text thread, but a lot depends on how complicated your layouts are and what else is on the spread. I strongly recommend starting from the front of the book, and manually setting the page #s for each chapter using the section options menu, so that you can decide which side of the spread each page is on. An odd numbered page will stay on the right side, and an even numbered page will stay on the left.