Navigating AI Privacy and Compliance in Healthcare

<p>hello everyone this is Cole from the American Journal of healthc care strategy I'm joined by a special guest today an expert in AI privacy in compliance Nico Nico please introduce yourself yeah hello my name is niik AD and I am now supposedly an expert [Music] I started my career through my undergraduate studies of Neuroscience and sociology at Welsley college that was actually what got me in interested in artificial intelligence because as I was in the process of graduating one of my last courses for so there's a couple of different um disciplines within Neuroscience the one that I was completing was specifically computational neuroscience and there was the option to take a course that was specific to artificial intelligence and even though it was just learning about neuron and the basis of how artificial intelligence would be developed how it's trying to be based off of how humans think but one thing that really St struck me when we had that particular course was that there was an interview that they had with joy bulam wami who was working at MIT she had a tech to Ted talk about how aut autonomous vehicles at that point in time had the potential of running over black people my class laughed because we're like oh this seems kind of ridiculous but she was very serious and said that no because of the way that these autonomous vehicles have been trained it seems as though they are struggling to recognize the humanity in certain types of people and then considering that was her entire work her entire thesis her thesis being gender Shades and how AI is cementing some of the biases that we have as humans it became my personal uh mission in life to make sure that doesn't affect marginalized communities and so through that experience I went and joined one of the labs that she had been part of while she was at MIT which was the data plus feminism lab she wasn't directly in this lab let me preface but one the principal investigator for this lab was one of her counselors so let I just wanted to clarify that on one side and so I learned a lot about data analysis and how important it is to have data tell narratives because data is just numbers whatever we get away from it is what we are able to tell a story from and then from there I worked at the sheller teachers education program also at MIT for AI education which come accumulated in making the makeon while I was doing both of those projects I met the founders of gradient Health which is where I currently work I am now a compliance officer for for a small medical AI company which focuses on creating large data off-the-shelf data sets for algorithm development that focuses on Health Equity that focuses on diversity and now as I continue to charge my own career path I'm now about to start my MBA and yeah I think that's a brief summary of my career at Absol incredible career just a few years out of your undergrad right like how many years are you out of undergrad now at this point I will now be for incred you've had such an incredible stretch and it's amazing the impact of just that Ted Talk that's had on you and I think that is motivation for me in doing this podcast is because you know you clearly already have the capability of motivating people just listening to your experience so far as motivational I want to ask why get that degree in Neuroscience that's a very challenging degree to attempt so you know four years ago or actually eight years ago at this point right when you started what made you want to start that degree I always was interested in the mind and initially I thought to myself that the biggest things that I found interesting about the mind is how can something biological be the basis of Consciousness how could that create the minds that caus us for me to communicate with you and for you to communicate back to me and then for all of the ways that humans create and build and do all the things that they do it was very difficult I have to give all of the shout outs to my advisor at the time her name was Dr Sarah waserman who was also a Welsley grad because I remember that my first Neuroscience class I had no idea what was going on I struggled so much in the first couple of courses that had to do with neuroscience and I remember my professor took me aside and said I recognize that you're struggling in the course but I understand that you understand what's happening here she took me aside and said that every single time we have a class I want us to have a session where you tell me what you understand about the course so I understand what your understanding and that built up my personal confidence to know that I could do this that I could keep going along in this course work even though it was extremely difficult because Neuroscience touches on biology chemistry in some cases physics a lot of physical mathematics not just in the way that the brain is formed but also in the ways in which chemical reactions happen in the brain you need a really strong mathematical background for that the labs I enjoy the labs greatly and then just understanding in all the ways that you have to have patience with yourself because not even the experts within Neuroscience know everything that's going on having the patience with yourself to be able to say like hey this is what I understand this is what I'm interested in let's keep going it's the way that you keep motivating yourself to complete the course and I again I have to give a shout out to all of the professors that I had while I was attending Wellsley because they were such a motivating team to push me to make sure that I knew that I could complete this I love sociology but sociology came a lot more EAS um and it was it it definitely I'm thankful that I did that course I tried and failed at even looking into majoring in Neuroscience during my time at the Department of Neurology so I am very impressed I worked with several MD phds who are neurology residents and just the amount of work focus on a single topic is just incredible so I am very impressed even by what you've done so far in our conversation but what is more interesting is after this degree program you get this you know you work in this lab at MIT and what's really interesting about that is yes you have this background in Neuroscience from Welsley which is impressive but there's lots of students in the area of Massachusetts which have these degrees even you know at MIT there's lots of students who can't get into these Labs how did you manage to get involved in this lab and and network with these people because that takes a completely different skill set on the networking end how did you manage to do that so the course that touched my heart so much on on AI we had Katherine deagazio who is the principal investigator for data plus feminism she came to give a talk at Welsley about all of the work that she's done she's been doing a lot of work on feminicide specifically in South America I thought it was amazing because I thought that most of the sciences that go on at MIT are hard Sciences like I thought that it was going to be a focused on just the pure biological facts of the matter I was so blown away that data plus feminism as a lab existed specifically in I think it's DSP which is one of part of the urban de um and planning uh sector within MIT because it had a social impact focus and so after the talk that she gave at Welsley I came up to her with so many different questions because I was more cognizant of the fact by this time in my degree I was now a senior this was before covid really truly hit so I think this was one of the last in-person conversations that I had I I didn't realize it at the time and I was talking to her about my concerns about how technology is going to have disperate impacts on marginalized communities and I kept in contact with her so when everything went virtual I reached out to her against S I recognize now that everything is difficult but I'm so moved by this particular problem if there's any work that you have for me anything at all please reach out to me please let me know if there's any work to do and Katherine is amazing Katherine has just published her second book I need to remember the name otherwise I don't want to butcher it but it's again on continuing her work on femicide and how it's a proliferating issue across um South America and even trying to see how that's a global issue and so that summer she was working on work for the com com com ah I'm so sorry commemoration Heritage landscape of the United States and this had to do specifically in the wake of George Floyd because of the Nexus that happened in the summer of 2020 where we were now being cognizant of the biases that exist in the police force in her lab you how to use technology to affect social impact I was brought on to work on that particular project and so I ended up managing a group of undergraduates who were focusing on how to use mapping to analyze different social issues specifically for how the birth of place of feminism how the United States in general works on specifically suffer jet movements and then in addition to that working on my own personal project which was black women remembered and recognizing how that had a huge impact through the black lives matter movement and its height in 2020 ironically I continue to use those spatial mapping techniques I learned that summer and other social impact volunteering work that I do now with Kansas City Defenders which is an abolitionist social impact room based in the midwest that is focusing on also figuring out why has there been a spike in missing persons reports in Missouri and why is that 4X what it was last year I learned about data narratives through this particular lab I learned about spatial technology and I also had a really supportive principal investigator that pushed me to know that I could do this work in addition to how important it is to marry technology with social impact and Community activism why do you think she chose you to do this work I have the habit of I don't want to say hunting people down but I really am overly enthusiastic when I think something's important even my current job I basically followed the founders around for six months for them to give me this job because I was blown away by the work that they were doing I think that I asked really difficult questions and I keep asking those questions and I am very very thankful that I've met a lot of people who have given me the opportunity to try and explore those answers how do you manage that the time it takes to be self-aware enough to ask those questions to reach out to these individuals while also pursuing such a hard degree how did you manage the time in that area are there any skills that you Ed that you could share I think that I've been a morning person my mom has made me into a morning person because we had to Growing Up I was raised um in Ghana so I went to an international school that was based in the city center while we lived about two hours out and so every morning we would wake up at 5: so that we could leave the house by 6 so we could be there by 8 and so that is something that has stuck with me throughout my adult life where I wake up really early in the morning and I get the hardest thing done first whether it's studying whether it's exercise whether it's sometime prayer or some sort of like spiritual feeding I get the hardest thing done first and then everything else comes later I focus on trying to make sure that I also build really strong communities where if I meet someone and I think that they're passionate about their own work I really want to cement connections like that because it make it's so important to think deeply about the work you're doing and not just do it for the sake of doing it I focus more on the passion around me instead of intelligence and I think that that is something that we sometimes take for granted it's important to be intelligent it's important to be curious don't get me wrong but it's even more important to say that I am the change that I want to see and I am going to make this work and because of that whenever I see a person who's trying to change the world in their own way whether it's in Katherine diagnos research or in great Health Mission I make sure that I stay around those people it's impressive because this you know anyone who would listen to you talking about your experiences with the lab I mean that's an incredible amount of experience that you gained there and it probably would have been challenging to gain that in any other way and so I think that I appreciate you sharing why you know what made you such a hard worker and give you those skills because other people are going to really want to adopt that so that they can get these opportunities as well so this lab experience set you up for your role with gradient how did that happen you touched on it earlier but can you walk us through exactly how you said that you chased them down for that role because you were inspired with what they were doing how did all of that come about so I had joined a group called Venture for America that brought me to Kansas City for about 3 years before I moved back to the east coast and while I was there I was working with another company like was basically Consulting for startups because the last year that there was a Tech Stars based in Kansas city was in 2021 and I had just moved to Kansas City at that time so I at the simultaneously while I was working with them was trying to like mulling over in my mind how can I use like how would I be able to use AI to address Healthcare issues how would I be able to use he AI specifically in confr in bias in the ways that we develop algorithms and gradient Health arrived as an answer we were supposed to consult with them to like they wanted feedback on their website and I remember giving them scathing feedback on their first iteration of their website because at that time they were just building the product it's not even something that was the most important thing for their product but I remember that the CEO his name is Josh Joshua Miller he laughed because he didn't realized that I would be so harsh on what they were presenting to the world I met them again when they were giving their presentation at Kansas City Tech Stars and I came to their Booth to go and ask them several questions that have been mulling over in my mind like for example why would radiological graphs while they're very useful for um development in algorithms is there any major difference between races when it comes to radio autographs it turns out there's not but there is a big difference in weight classes and so I was asking them that oh how do you account for weight classes especially since um Americans are on the heavier side considering at this time they were sourcing a lot of their data out of Southeast Asia and they realized that oh we hadn't actually considered that and I said yes in addition to the fact and I'm certain they knew this but they were blanking at the time that for the FDA they would want to prioritize American data which they ended up prioritizing in addition to also trying to in an effort to continuing to further their social impact they considered uh collecting data from all over the world whether it was in Brazil whether it was in Africa they continued to build out these relationships because I was also having these conversations with them and continuing to have these conversations with them over email they eventually raised their first seed round and I went and reached out to Josh about like hey I T I gave you this Consulting advice I came and found you guys at this then I keep emailing you do you have a job available for me and I became their first employee they didn't really have any structure back in those days at all it's funny because I since I was the first employee I was responsible for building out a lot of structure that exists in the company right now from starting out as a marketing analyst to trying to get into sales and then I took a sabatical so I could become a stronger software engineer because even though I had some software engineering experience most of it had nothing to do with web development and then coming back and realizing that we also need to build out our security protocol since we're dealing with really really sensitive data and acting for us as a privacy analyst and then eventually after I completed our first stock 2 audit with an unqualified opinion which is the highest opinion that you can get on one of these things I was now promoted to compliance officer and that is the role that I've been acting in primarily for the past year I just bother people a lot until they like you know what here do this and then we go from there where do you think now well now that you've been in this role with this organization for a while now you've gained some knowledge where do you think we're going with AI privacy legislation and what needs to happen to protect everyone equally as we know privacy legislation usually does not protect everyone equally so what needs to happen to make sure that every 's protected equally in the AI space I'm really excited about the legislation that's been enacted in New York and I think is coming in New Jersey where the primary motive of this is to be able to opt out of AI Services we're getting to the point where AI has developed to the level that chat Bots are replacing humans for customer service and sometimes you just want to talk to a human I think it's really important to have the op option to continue to interact with humans especially as we continue to iron out the issues that exist in AI without a doubt the AI that has developed over the past two to three years through language models through chat Bots is impressive but there's so much lacking in it that we need to at least be able to be accountable to real humans before we can start rolling it out on a larger scale I think that the first thing that most policies can do is to make sure that it's still an option to be able to escalate to talk to an actual person the second thing that I think is really important in AI policies is to make sure that we are continuing to research a large breath of the population and not focusing on one particular type of data set ironically radi in health is now currently working with um a federal contract that is supposed to make sure that we can audit their algorithms one thing that a lot of algorithms can do is that they test really well with their own data sets where they can get 90 95% accuracy but the moment you test them on some random data set of the same type it doesn't know what it's doing we're trying to make it so that you should be able to audit algorithms it is so important to be able to explain what something is doing to the general public the third thing that I think is really important is to have General AI education on the entire societal level level I think it's now imperative that as part of stem courses specifically um computer science courses even if coding is not involved it is really fundamentally important to be able to say that hey this is how algorithms are developed this is how they process your data this is how the data talks back to you and this is what it can and cannot do I think it's really important to make sure that students are aware of how that works especially so that they don't think it's magic it was so eye openening to when I was working with the step Lab at MIT where we were working on the daily um AI curriculum which was targeting children based from 9 to 18 where a lot of students were not aware that even the way that you interact with recommendation systems in Netflix in in most streaming services even on um using Google search that those are forms of machine learning and that those forms of AI are not perfect that there are sometimes bias that exist in those and that it builds upon itself while recommendation systems are building upon your data what you enjoy what you find interesting sometimes when you're using um other algorithms like Google's NLP algorithm certain terms and keywords will bring up racist sexist examples or just misinformation that is being spread not because it's a malicious algorithm remember algorith rithms aren't real like real people they're not autonomous at this moment in time but because of what we've used to code into it and it was really heartening to see that the children were like oh no this is terrible but also like realizing that they should take some of the things that they search with a grain of salt once people are more aware of that it makes them more confident in interacting with these systems and it is interesting with that and you know algorithms these AI systems are essentially just really advanced mathematics at least a lot of the simpler ones are I know at my health plan for the AI to determine how much something is going to cost it's just complicated equations essentially I want to ask with that complicated equation though have you you've probably heard of this they had this automated detection AI system in their cameras to flag potential shoplifters and trouble people as they said and then to have them kicked out seemed like it only alerted to female people of color pretty much it and nobody else how do we you know camera technology has had racial issues for a very long time since the iPhone x came out iPhone 10 came out it was with the portrait mode it only work with white people and it's gotten better thankfully but that was many generations later how do we as an institutions like gradient for example put policies in place to prevent rolling out technology olog that is racist or exhibits bias in some way I think that if we think the bias is bad enough once we have examples of it take it off the shelves Joy bwami rolled out a campaign that succeeded in the city of Cambridge in Massachusetts to take down all measures of facial recognition software because at that point in time there was a measurable difference between the facial recognition software ability to identify women of color and specifically black women of color where I think white males had an accuracy rate of 98 99% while um black W black females had a accuracy rate of like 82% and even within that those per huge difference in percentage points that identifying people incorrectly using facials recognition technology was so egregious that the only responsible thing to do was to take it back a notch there is nothing wrong with continuing to have research to improve these Technologies but especially specifically in the case for facial recognition software being in a surveillance capitalism state is not safe for majority of people across the board it does not particularly make Community safer it causes undue stress and in addition to that it's just not correct a lot of the times where people were like you the example that you brought up where people were wrongfully being accused of shoplifting by the time these things are rectified people have lost wages from potentially being fired from their um employers they've had their name and reputation called into question at certain points those things are not worth pursuing gradient Health one of the things that I'm really happy about is that we currently do not work with patient um live patient data all of the data that we're working on is specifically for research purposes because until we're in the 99 percentile we shouldn't be saying that hey we can predict with 90 90% accuracy that you have cancer until we are at a point where we can say yes this is exactly what we that we are confident that we can do I am proud to continue to support research but I think that sometimes we are a little too enthusiastic about releasing research projects into the wild I think that sometimes we should slow down and make sure that there's more work to be done before we allow people to interact with that absolutely I I completely agree and going back to what we've known throughout history in the United States and across the world is that when there's a group of people who are unsafe or a group of people who are lacking Justice everybody is lacking Justice and everybody is unsafe it creates a unsafe Society in general and so hopefully we with people like yourself we will continue to prioritize this technology being equal and lacking bias hopefully so thank you so much for coming on Nico I couldn't be more appreciative I really hope we can have you back on again as you continue your career in your journey thank you so much for having me</p>