Can you confirm from your brother what exactly CIC has on record? If you were listed anywhere on his application then you may have a chance, if otherwise then I doubt you can claim points.
According to the courts, “The effect of the final decree of adoption is to, “terminate all legal relationships between the adopted individual and his natural relatives, including his natural parents, so that the adopted individual thereafter is a stranger to his former relatives for all purposes.”
According to CIC:
For both the immigration and citizenship process, an adoption is valid only if it ends the child’s legal relationship with the biological parents, and a new legal relationship is established between the adopting parents and the adopted child.
This means the child cannot later sponsor his/her biological relatives to come to Canada.
If your brother is in Manitoba, you can try the PNP route.